Data & Databases
Intermediate
12 chapters
Free certificate

Data Engineering & The Modern Data Stack

Pipelines, columnar engines, Airflow DAGs and dbt models, visualised.

Before anyone builds a dashboard, data has to be moved, cleaned and loaded. You will watch pipelines flow like water through pipes.

Your progress (saved in this browser)0/12

Try it in 10 seconds

Every chapter teaches like this: guess first, then see why.

Predict first

A pipeline re-runs after a crash. Running it twice should…

Your roadmap

  1. Chapter 1

    What is Data Engineering? Building the Data Highways

    Learn the plumbing of big data: the role of data engineers, the 3-stage ETL pipeline (Extract, Transform, Load), OLTP vs OLAP databases, and Apache Parquet files.

  2. Chapter 2

    Batch Processing vs. Real-Time Streaming

    Master how enterprise data moves: scheduled Batch pipelines vs millisecond real-time Streaming, cost and latency tradeoffs, and the crucial rule of idempotency.

  3. Chapter 3 · coming soon

    File Formats Deep Dive: CSV vs. JSON vs. Parquet (Column Pruning & Compression)

  4. Chapter 4 · coming soon

    ETL vs. ELT: Why Modern Cloud Data Stacks Load First and Transform Later

  5. Chapter 5 · coming soon

    Transformation with dbt & SQL Models

  6. Chapter 6 · coming soon

    Orchestration & Directed Acyclic Graphs (DAGs) with Apache Airflow

  7. Chapter 7 · coming soon

    Data Quality, Schema Evolution & Anomaly Alerts

  8. Chapter 8 · coming soon

    Message Brokers & Event Streams: Kafka & RabbitMQ Concepts

  9. Chapter 9 · coming soon

    Distributed Compute Fundamentals: MapReduce & Spark Partitions

  10. Chapter 10 · coming soon

    Data Lakes, Warehouses & Lakehouses (Delta Lake / Iceberg)

  11. Chapter 11 · coming soon

    Reverse ETL & Serving Data to Production APIs

  12. Chapter 12 · coming soon

    Data Engineering Capstone & System Design Interview

Related courses