dstandish edited a comment on issue #14338: URL: https://github.com/apache/airflow/issues/14338#issuecomment-783496230
the dags (where the subdags lived) had a dag concurrency of 1. it wasn't general scheduler concurrency (i.e. cluster parallelism or whatever it is) but so it sounds like you are saying these simply needed to be updated to have a concurrency of 2.... for example. these seems a bit risky what if after finishing first task in a subdag the scheduler launches a new subdag instead of another task in the first subdag.... not that this happens just thinking it through. anyway i don't see any change to concurrency handling in https://github.com/apache/airflow/blob/master/UPDATING.md .... and these subdag dags did run properly before upgrading, (though there is mention of scheduler vs backfill) so it seems there may be something undesirable going on. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
