Alberto Anceschi created AIRFLOW-3244:
-----------------------------------------
Summary: Introduce offset on the execution date for data assessment
Key: AIRFLOW-3244
URL: https://issues.apache.org/jira/browse/AIRFLOW-3244
Project: Apache Airflow
Issue Type: Improvement
Components: DAG
Affects Versions: 1.10.0
Reporter: Alberto Anceschi
Hi everyone,
I'm trying to port my current cronjobs into Airflow. Let's consider a real case
scenario: I've to send every week a report and through the pipeline data from
Google Analytics needs to be collected, so I need 2 days before running the DAG
(data assessment). Week starts on Monday and ends on Sunday, so I need the DAG
to run on Wednesday at Midnight UTC.
In order to see on the Airflow dashboard start_date/exection_date that make
sense to me, for now I've used a TimeDeltaSensor that adds that 2 day offset I
need, but this is not its purpose. I also use Celery executor, so its workers
keep polling during those 2 days, making them unavailable for other DAGs.
I think that the assumption that at the end of the period scheduled data are
ready is not correct and at the same time it's much more intuitive seeing on
the dashboard Monday execution dates instead of Tuesday ones.
What do you think about this request? Any suggestion? Thank you,
Alberto
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
