|
Writing production-ready ETL pipelines in Python / Pandas 2022-4
Description
Writing production-ready ETL pipelines in Python / Pandas is the name of the training course that will teach you every step of writing an ETL pipeline in Python from the beginning to production using the necessary tools such as Python 3.9 and Jupyter Notebook and It will show Git, Github, Visual Studio code, Docker, Docker Hub, and Python packages including Pandas, boto3, pyyaml, awscli, jupyter, pylint, moto, coverage, and memory profiler. Two different coding approaches have been introduced and applied in the field of data engineering, including functional and object-oriented programming.
The best methods in Python code development have been introduced and applied, including design principles, clean coding, virtual environments, project/folder setup, settings, logging module, exception and error management (Exception Handling), linting tools, dependency management tools. or dependency management, tuning and performance optimization using profiling, unit testing module, integration testing tool and dockerization tool.
Things you will learn in this course:
- How to write professional ETL Pipelines in Python
- Steps to write production-level Python code
- How to apply functional programming in data engineering
- How to design proper object oriented code
- How to use meta file for job control
- Coding best practices for Python in data engineering/ETL
- How to implement a pipeline in Python to extract data from an AWS S3 source and convert and load data to another AWS S3 target.
This course is suitable for people who:
- Data engineers, scientists, and developers who want to write professional production-ready data pipelines in Python.
- Anyone interested in writing production-ready data pipelines in Python.
Specifications of the Writing production-ready ETL pipelines in Python / Pandas course:
- Publisher: Udemy
- Teacher: Jan Schwarzlose
- English language
- Education level: from introductory to advanced
- Duration: 7 hours and 3 minutes
- Number of courses: 78
|
