
goals

- dags, operators, hooks
- airflow ui
- how internally
- execution date, upstream/downstream
- debug

pro compared to crontab
- dependencies between tasks
- run parallel
- rerun specific tasks
- separate logs

DAG
- directed acyclic graph
- one direction
- nodes and edges
- no loops

when airflow
- complex pipelines
- only batch
- only orchestration / no processing

concepts
- scheduler
- dags
- workers
- operator
  - many operators / default or write your own

draw
https://excalidraw.com

schedule interval
https://crontab.guru


start date
once interval is finished
@daily will start to run
set up your code so it can use execution_date from airflow

possible dependency: only when day before has completed


every three days
0 */72 * * *

https://airflow.apache.org/docs/apache-airflow/stable/templates-ref.html

https://www.udemy.com/course/the-complete-hands-on-course-to-master-apache-airflow/