Audience
This course can be beneficial for various professionals, including: developers, analysts, software programmers, consultants, data engineers, data scientists , data analysts, software engineers, Big data programmers, Hadoop developers
Prerequisites
Delegates must have a basic idea of a programming language and a development framework.
Duration
4 days.
Course Objectives
PySpark is an interface for Apache Spark in Python. With PySpark, you can write Python and SQL-like commands to manipulate and analyze data in a distributed processing environment.
Course Content
Python Quick Start
Python Essentials
Language Fundamentals
Functions
Data Structures
Comprehensions
Introduction to Big Data
Setting the Scene
Introduction to Hadoop
Hadoop Components
Getting Started with PySpark
Introduction to Spark
Spark Architecture
Application Execution
Using the Python Spark Shell
Using the PySpark API
Essential Concepts
Creating an RDD
Working with RDDs
RDD Operations - Part 1
RDD Transformations
RDD Transformations on Key-Value Pairs
RDD Operations - Part 2
Actions
Caching
Spark Jobs - The Big Picture
Getting Started with Spark SQL
Overview of Spark SQL
Getting Started with the Spark SQL API
Creating DataFrames from a Data Source
Creating DataFrames from an RDD
Spark SQL DataFrame Operations
Basic Operations
Language-Integrated Query Operations
RDD Operations
Output Operations