One of those days ... max_active_runs is a dag property. default_args only get passed as default args to task instances, but it never applies there.
Thanks! G> On Wed, Feb 14, 2018 at 2:47 PM, Ash Berlin-Taylor < [email protected]> wrote: > It seems unlikely, but could it be the location of where max_active_runs > is specified? In our DAGs we pass it directly as an argument to the DAG() > call, not via default_arguments and it behaves itself for us. I think.... > I should check that! > > -ash > > > > On 14 Feb 2018, at 13:43, Gerard Toonstra <[email protected]> wrote: > > > > A user on airflow 1.9.0 reports that 'max_active_runs' isn't respected. I > > remembered having fixed something related to this ages ago and this is > > here: > > > > https://issues.apache.org/jira/browse/AIRFLOW-137 > > > > That however was related to backfills and clearing the dagruns. > > > > I watched him in the scenario and he literally creates a new simple dag > > with the following config: > > > > ----- > > > > from airflow import DAG > > from datetime import datetime, timedelta > > > > from airflow.contrib.operators.bigquery_operator import BigQueryOperator > > from airflow.contrib.operators.bigquery_to_gcs import > > BigQueryToCloudStorageOperator > > from airflow.contrib.operators.gcs_download_operator import > > GoogleCloudStorageDownloadOperator > > from airflow.contrib.operators.file_to_gcs import > > FileToGoogleCloudStorageOperator > > from airflow.operators.python_operator import PythonOperator > > from airflow.models import Variable > > import time > > > > default_args = { > > 'owner': 'airflow', > > 'start_date': datetime(2018, 2, 10), > > 'max_active_runs': 1, > > 'email_on_failure': False, > > 'email_on_retry': False, > > } > > > > dag = DAG('analytics6', default_args=default_args, schedule_interval='15 > 12 > > * * *') > > > > ----- > > > > When it gets activated, multiple dagruns are created when there are still > > tasks running on the first date. > > > > His version is 1.9.0 from pypi. > > > > Is max_active_runs broken or are there other explanations for this > > particular behavior? > > > > Rgds, > > > > Gerard > >
