Data Pipelines and Apache Airflow

Apache Airflow is a batch-oriented framework used building data pipelines. The key features are.

Data pipelines are are group or arrangement of tasks or actions that need to be executed to achieve the fetch transform and deliver data.

Main concepts

In airflow data depencies are represented as a graph.