GitHub user dstandish closed the discussion with a comment: Should you be able to override pools in backfill?
Responses from slack: > > raphaelauv > Yesterday at 1:22 PM > Backfill was ignoring pools , that's a bug > Backfill is a trick on time , not on the code of the dag , airflow should > respect the code of the dag no matter what > > > Daniel Standish > [Yesterday at 1:24 > PM](https://apache-airflow.slack.com/archives/C06K9Q5G2UA/p1729455867348749?thread_ts=1729436754.752759&cid=C06K9Q5G2UA) > And your opinion concerning a possible interface for overriding pools? > [1:24](https://apache-airflow.slack.com/archives/C06K9Q5G2UA/p1729455889539649?thread_ts=1729436754.752759&cid=C06K9Q5G2UA) > Maybe it’s not really necessary, or not as important, now that we will have > separate concurrency controls for backfill? > > > Jarek Potiuk > Yesterday at 2:54 PM > Yeah. Backfill ignoring pools was a bug. > I think we should not override pools at all with backfill - see the discusion > on backfill vs. deferrable. There is a reason a task needs a pool, and > backfill ignoring that is a bad idea. (edited) > :heavy_plus_sign: > 2 > > [2:56](https://apache-airflow.slack.com/archives/C06K9Q5G2UA/p1729461380822389?thread_ts=1729436754.752759&cid=C06K9Q5G2UA) > There are probably a couple of “override” scenarios, but - paraphrasing your > request > [@dstandish](https://apache-airflow.slack.com/team/UG7RJQEV9) > from the backfill/deferrable discussion - we should only allow > overridability if someone comes up with - maybe a made up but realistic - > case where it makes sense (edited) > > > raphaelauv > Yesterday at 5:05 PM > a possible interface for overriding pools > maybe some people would expect this feature ( and I don't see what are the > purpose of pools in that case in this scenario ) , but they can always > copy paste the dag > sufix the name > edit the pool usage in the tasks > trigger the backill of the dag > > > Daniel Standish > [6 minutes > ago](https://apache-airflow.slack.com/archives/C06K9Q5G2UA/p1729539875337429?thread_ts=1729436754.752759&cid=C06K9Q5G2UA) > and I don't see what are the purpose of pools in that case in this scenario > [12:44](https://apache-airflow.slack.com/archives/C06K9Q5G2UA/p1729539888781359?thread_ts=1729436754.752759&cid=C06K9Q5G2UA) > pools are a bit of an enigma > [12:45](https://apache-airflow.slack.com/archives/C06K9Q5G2UA/p1729539918844969?thread_ts=1729436754.752759&cid=C06K9Q5G2UA) > alright, well, if people say no override, then no override there shall be > [12:45](https://apache-airflow.slack.com/archives/C06K9Q5G2UA/p1729539926740689?thread_ts=1729436754.752759&cid=C06K9Q5G2UA) > i will make a lazy consensus vote > > > raphaelauv > 4 minutes ago > pools are a bit of an enigma > I only use them as cross-dag semaphore > > GitHub link: https://github.com/apache/airflow/discussions/43235#discussioncomment-11009376 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
