Airflow is a platform created by community to programmatically author, schedule and monitor workflows.
It is scalable, dynamic, extensible and modulable.
Without any doubts, mastering Airflow is becoming a must–have and an attractive skill for anyone working with data.
What you will learn in the course:
Fundamentals of Airflow are explained such as what is Airflow, how the scheduler and the web server works
The Forex Data Pipeline project is incredible way to discover many operators in Airflow and deal with Slack, Spark, Hadoop and more
Mastering your DAGs is a top priority and you will be able to play with timezones, unit testing your DAGs, how to structure your DAG folder and much more
Scaling Airflow through different executors such as the Local Executor, the Celery Executor and the Kubernetes Executor will be explained in details. You will discover how to specialise your workers, how to add new workers, what happens when a node crashes.
A Kubernetes cluster of 3 nodes will be set up with Rancher, Airflow and the Kubernetes Executor in local to run your data pipelines.
Advanced concepts will be shown through practical examples such as templatating your DAGs, how to make your DAG dependent of another, what are Subdags and deadlocks, and more.
Courses : 2
Specification: Apache Airflow: The Hands-On Guide
13 reviews for Apache Airflow: The Hands-On Guide