Users Online

· Guests Online: 13

· Members Online: 0

· Total Members: 188
· Newest Member: meenachowdary055

Forum Threads

Newest Threads
No Threads created
Hottest Threads
No Threads created

Latest Articles

Writing production-ready ETL pipelines in Python / Pandas 2022-4

Writing production-ready ETL pipelines in Python / Pandas course

Created by Jan Schwarzlose


05_033 s3.py

Writing production-ready ETL pipelines in Python / Pandas 2022-4

 

 

Description

Writing production-ready ETL pipelines in Python / Pandas is the name of the training course that will teach you every step of writing an ETL pipeline in Python from the beginning to production using the necessary tools such as Python 3.9 and Jupyter Notebook and It will show Git, Github, Visual Studio code, Docker, Docker Hub, and Python packages including Pandas, boto3, pyyaml, awscli, jupyter, pylint, moto, coverage, and memory profiler. Two different coding approaches have been introduced and applied in the field of data engineering, including functional and object-oriented programming.

The best methods in Python code development have been introduced and applied, including design principles, clean coding, virtual environments, project/folder setup, settings, logging module, exception and error management (Exception Handling), linting tools, dependency management tools. or dependency management, tuning and performance optimization using profiling, unit testing module, integration testing tool and dockerization tool.

Things you will learn in this course:

  • How to write professional ETL Pipelines in Python
  • Steps to write production-level Python code
  • How to apply functional programming in data engineering
  • How to design proper object oriented code
  • How to use meta file for job control
  • Coding best practices for Python in data engineering/ETL
  • How to implement a pipeline in Python to extract data from an AWS S3 source and convert and load data to another AWS S3 target.

This course is suitable for people who:

  • Data engineers, scientists, and developers who want to write professional production-ready data pipelines in Python.
  • Anyone interested in writing production-ready data pipelines in Python.

Specifications of the Writing production-ready ETL pipelines in Python / Pandas course:

  • Publisher: Udemy
  • Teacher: Jan Schwarzlose
  • English language
  • Education level: from introductory to advanced
  • Duration: 7 hours and 3 minutes
  • Number of courses: 78

      
Course Contents
01 Introduction 02 Quick and Dirty Solution 03 Functional Approach 04 Object Oriented Approach 05 Setup and Class Frame Implementation 06 Code Implementation 07 Finalizing the ETL Job 08 Summary

Comments

No Comments have been Posted.

Post Comment

Please Login to Post a Comment.

Ratings

Rating is available to Members only.

Please login or register to vote.

No Ratings have been Posted.
Render time: 0.71 seconds
10,923,021 unique visits