milton0825 commented on a change in pull request #4949: [AIRFLOW-1557] Backfill should respect pool limit URL: https://github.com/apache/airflow/pull/4949#discussion_r268257800
########## File path: airflow/config_templates/default_airflow.cfg ########## @@ -125,6 +125,10 @@ dags_are_paused_at_creation = True # whose size is guided by this config element non_pooled_task_slot_count = 128 +# When not using pools, the number of backfill tasks per backfill +# is limited by this config element +non_pooled_backfill_task_slot_count = 128 Review comment: @ashb Let me rephrase it a bit. In some cases, people do not run backfills on where scheduler runs... Anyway I think we should still add this config so that people can fine tune the slot count if they launch backfill job on a machine other than scheduler ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
