Ruslan Dautkhanov created AIRFLOW-1178:
------------------------------------------
Summary: @once may run more than one time
Key: AIRFLOW-1178
URL: https://issues.apache.org/jira/browse/AIRFLOW-1178
Project: Apache Airflow
Issue Type: Bug
Components: scheduler
Affects Versions: Airflow 1.8, 1.8.1, 1.8.0
Environment: Airflow master snapshot from May 05 2017
Reporter: Ruslan Dautkhanov
Priority: Critical
Attachments: onceDAG_got_scheduled_twice.png
My DAG is running second (2nd) time although it is declared as @once.
Here's DAG definition :
{noformat}
main_dag = DAG(
dag_id = 'Test-DAG-1',
default_args = default_args, # dafeult
operators' arguments - see above
user_defined_macros = dag_macros, # I do not get different
between
## params = dag_macros, # user_defined_macros
and params
#
start_date = datetime.now(), # or e.g.
datetime(2015, 6, 1)
# 'end_date' = datetime(2016, 1, 1),
catchup = True, # Perform
scheduler catchup (or only run latest)?
# -
defaults to True
schedule_interval = '@once', #
'@once'=None?
# doesn't
create multiple dag runs automatically
concurrency = 3, # task
instances allowed to run concurrently
max_active_runs = 1, # only one
DAG run at a time
dagrun_timeout = timedelta(days=4), # no way
this dag should ran for 4 days
orientation = 'TB', # default
graph view
)
{noformat}
As a workaround for AIRFLOW-1013 I changed catchup from False to True.
Suggested on dev list. It "worked around" AIRFLOW-1013 execution, but screwed
@once logic - the DAG got scheduled twice (!) which is a no-go for us. The DAG
actually has to run not more than 1 time.
IMO, catchup=True should be explicitly disallowed for @once schedule.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
