PySpark

Section 1: Big Data Analytics introduction

 Big Data summary

 Features of Apache Spark

 Use Cases of Apache Spark

 Spark Execution

 Job Execution Flow

 Why Spark with Python

 Apache spark Architecture

 Big Data Analytics in business

Section 2: Using Hadoop’s Core: HDFS and MapReduce

 HDFS and how it operates

 MapReduce how it operates

 How MapReduce categorizes processing

 HDFS commands

Section 3: Spark Databox Cloud Lab

 How to access SparkDatabox cloud lab?

 Step by Step instruction to access cloud Big data Lab.

Section 4: Data analytics lifecycle

 Data Discovery

 Data Preparation

 Data Model Planning

 Data Model Building

 Data Insights

Section 5: Python 3.0 ( Crash Course )

 Environment Setup

 Decision Making

 Loops and Number

 Strings

 Lists

 Tuples

 Dictionary

 Date and Time

 Regex

 Functions

 Modules

 Files I/O

 Exceptions

 MultiThreading

 Set

 Lamda Function

Section 6: PySpark

 Introduction to SparkContext

 Environment Setup

 Spark RDD

 spark Caching

 Common Transformations and Actions

 Spark Functions

 Key-Value Pairs

 Aggregate Functions

 Working with Aggregate Functions

 Joins in Spark

 Spark DataFrame

Section 7: Advanced Spark Programming

 Spark Shared Variables

 Custom Accumulator

 Spark and Fault Tolerance

 Broadcast variables

 Numeric RDD Operations

 Per-Partition Operations

Section 8: Running Spark jobs on Cluster

 Spark Runtime Architecture

 Spark Driver

 Executors

 Cluster Managers

 Connecting Spark To Different File System and Perform ETL (Extraction Transformation and Loading)

 Connecting Spark To DataBases and Perform ETL (Extraction Transformation and Loading)

 Spark StorageLevel

 Spark Serializers

 Spark-Submit and Cluster Explanation

 Performance Tuning

Section 9: PySpark Streaming at Scale

 Spark Streaming

 PySpark Streaming with Apache Kafka

 Real-world Practical use cases

 Operations On Streaming Dataframes and Datasets

 Window Operations

Section 10: Real-world project training

 PySpark project environment setup

 Real-world PySpark project

 Project demonstration