PySpark Python Spark Hadoop coding framework and testing
Big data Python Spark PySpark coding framework logging error handling unit testing PyCharm PostgreSQL Hive data pipeline
This course will bridge the gap between your academic and real world knowledge and prepare you for an entry level Big Data Python Spark developer role. You will learn the following
-
Python Spark coding best practices
-
Logging
-
Error Handling
-
Reading configuration from properties file
-
Doing development work using PyCharm
-
Using your local environment as a Hadoop Hive environment
-
Reading and writing to a Postgres database using Spark
-
Python unit testing framework
-
Building a data pipeline using Hadoop , Spark and Postgres
Prerequisites :
-
Basic programming skills
-
Basic database knowledge
-
Hadoop entry level knowledge