bperson opened a new pull request #15247: URL: https://github.com/apache/airflow/pull/15247
Should fix #14515. I saw 3 ways of fixing it: 1. do a DB migration to represent an infinite pool with `maxint` slots instead of `-1`. 2. do the same thing but at runtime in `slots_stats`. 3. get `_executable_task_instances_to_queued` to understand `-1` pools, changing all code operating on `pool_slots_free` and `open_slots ` [#](https://github.com/apache/airflow/blob/master/airflow/jobs/scheduler_job.py#L908) [#](https://github.com/apache/airflow/blob/master/airflow/jobs/scheduler_job.py#L975) [#](https://github.com/apache/airflow/blob/master/airflow/jobs/scheduler_job.py#L992) [#](https://github.com/apache/airflow/blob/master/airflow/jobs/scheduler_job.py#L1045) [#](https://github.com/apache/airflow/blob/master/airflow/jobs/scheduler_job.py#L1060) As to why I chose 2: 1. could be a decent alternative but it's a DB migration with all the complexities inherent to it. 2. is relatively non-intrusive: it's almost free to do it at runtime and it keeps the change completely in the core Python part of Airflow ( in case someone out there directly creates pools by inserting them in his DB 😨 ) 3. has the benefit of truly handling infinite pools but I doubt anyone has reached a scale where it matters ( we're talking about a pool with `9223372036854775807` slots here on my system ), plus it increases the overall complexity of that function which is close to the core of the scheduler, we might as well try to keep it as lean as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
