Course outline on Big Data for beginners using Hadoop

Course outline on Big Data for beginners using Hadoop

This 7 day course jumps into the world of Hadoop ecosystem components and its tools in a simplified manner, and provides you with the skills to utilize them effectively for faster and effective development of Big Data projects.

Course Outcome

Every learner has different learning patterns and absorption rates. As such, it is virtually impossible to gauge an exact learning outcome of each participant. However, it may be safely assumed that by the end of this course, each learner shall be able to:

• Understand how Hadoop clusters are managed by YARN, Tez, Zookeeper, Zeppelin,
• Install and run Apache Hadoop and its components on a cluster
• Frame big data analysis problems
• Implement iterative algorithms such as breadth-first-search
• Use DataFrames and Structured Streaming
• Use Spark's Resilient Distributed Datasets to process and analyze large data sets across many CPU's
• Understand how Spark Streaming lets your process continuous streams of data in real time
• Share information between nodes
• Analyze non-relational data using HBase, Cassandra, and MongoDB
• Publish data to your Hadoop cluster using either Kafka, Sqoop, or Flume
• Use Spark to create scripts to process data on a Hadoop cluster in more complex ways.
• Analyze relational data using Hive
• Python Programming
• Python for Spark
• Use the Linux operating system for setup of clusters
• Understand basic security systems pertaining to Big Data

Prerequisites

It is expected that the pupil shall have the following at hand when participating in the course:

• Ability to comprehend the English Language (basic)
• Access to High Speed Internet
• Access to Linux based operating system (at least 3 units)
• Basic understanding of OS
• Basic understanding and experience using the internet
• Basic understanding of filing systems
• High School Mathematics

Course Outline

1. Prerequisites
   1. Programming
      1. Python
         1. Introduction
         2. IDE
         3. Variables
         4. Data Types
         5. Conditions
         6. Loops
         7. Functions
         8. Recursion
         9. File Systems
         10. Database
      2. Java
         1. Introduction
         2. IDE
         3. Variables
         4. Data Types
         5. Conditions
         6. Loops
         7. Functions
         8. OOP
         9. Deployment
   2. OS and Networking
      1. Terminal
      2. SSH
      3. Linux Commands
      4. Linux Scripts
      5. Linux file system
      6. Keys
      7. Bash and profiles
2. Hadoop
   1. Introduction
   2. Background in Big Data
   3. Overview of components
   4. Installation and multi-node cluster setup
   5. MapReduce
   6. Ambari
   7. Yarn
3. Pig
   1. Installation
   2. Pig Latin
   3. Use Case
4. Hive
   1. Installation
   2. Queries
   3. Hive Partitioning, Buckets, UDFs, and SerDes
5. Kafka
   1. Installation
   2. Single node setup
   3. Multi-node setup
   4. Multi-node cluster with replication
   5. Programming for Kafka
      1. Python
      2. Java
6. Spark
   1. Spark Execution Model Architecture
   2. Programming in Spark with python
   3. Structured API foundation
   4. Dataframes
   5. Sinks
   6. Transformations
   7. Aggregations
   8. Joins
7. Hadoop Security
   1. Introduction to Kerberos
   2. Kerberos on Hadoop
   3. Kerberos Terminology
   4. Demo: Enabling Kerberos