·

Project 07_ETL_Pipeline

I developed a nightly ETL pipeline using Apache Airflow and Python. Airflow orchestrated the workflow and triggered a Python ETL script every night. The script connected to an on-premises PostgreSQL…

I developed a nightly ETL pipeline using Apache Airflow and Python. Airflow orchestrated the workflow and triggered a Python ETL script every night. The script connected to an on-premises PostgreSQL database using SQLAlchemy, extracted order data into a Pandas DataFrame, removed duplicates, handled missing values, validated records, converted date formats, and calculated business metrics such as total sales. After transformation, the cleaned data was loaded into an Postgr Database used for reporting.

Comments

4 responses

Leave a Reply

Your email address will not be published. Required fields are marked *