GitHub user raphaelauv added a comment to the discussion: Airflow Calendar: A
plugin to transform cron expressions into a visual schedule
this is very nice !!! very impressive works
in all case I would recommend to always start pipelines at midnight to have
"clean" data_interval_start and data_interval_end
and if there is a reason to wait some "times" before running tasks then add a
TimeDeltaSensor at the beginning of your dag
```python
waiter = TimeDeltaSensor(
task_id=f"wait_delay_XXXXXX",
delta=timedelta(hours=3), # in practice XXXXXXXXX data is not
available before XXX hours
poke_interval=timedelta(hours=1),
deferrable=True
)
```
and if multiple dags/tasks are impacted when running at the same time then use
pools to limit contention
GitHub link:
https://github.com/apache/airflow/discussions/64819#discussioncomment-17736528
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
