☁️ AWS Data Engineering • S3 • Glue • Lambda • Redshift • Airflow

AWS Data Engineering

End-to-end AWS lessons from the provided AWS lessons.txt — S3 features & classes, security, Glue DynamicFrames & DPU, Lambda triggers, SCD, Redshift distribution & sort keys, Airflow DAGs, all without overwriting SQL/PySpark.

Start with S3 → Explore Glue

Learning Path

8 chapters derived directly from AWS lessons.txt

🪣 01

S3

Features, Classes, Security, Versioning, Folder Structure, Cross-Account

Explore →

🔧 02

Glue

Catalog, DynamicFrames, DPU Workers, Crawlers, Jobs, Workflows, Bookmarks

Explore →

λ 03

Lambda

Serverless, Layers, Limitations, Boto3, Glue Triggers

Explore →

🕰️ 04

SCD Types

SCD 0-6 with PySpark example for SCD2

Explore →

🔄 05

ETL & Incremental

ETL vs ELT, Medallion, Incremental Loads, CDC/DMS

Explore →

🏢 06

Redshift

MPP, Columnar, Spectrum, Distribution, Sort Keys, Optimization

Explore →

🌬️ 07

Airflow

Operators, Sensors, DAG, Catchup/Depends, vs Step Functions

Explore →

🔐 08

DevOps

Git/GitHub, Secrets Manager, Status Codes

Explore →