GitHub user dstandish edited a discussion: Should you be able to override pools in backfill?
Note: I originally posted this on slack but I'm reposting here for posterity So i was going to implement the "pool override" behavior on backfill. But it strikes me that, the option to **provide one "pool override" that overrides pool for all tasks in the backfill**, that just doesn't make a lot of sense. Obviously different tasks may use different pools. so all-or-nothing override doesn't make a lot of sense. And note, in **the old backfill, it just straight up ignored pools unless you provided an override** --- by default just no pools were used. this was surprising to me. But ok, i don't really like that option. so what's the alternative? we could **allow user to provide a dict of tasks or something** where you optionally override the pool for any specific task. and then if no override is provided, then use the normal pool. this would work and make more sense. but it seems like a rather kludgy interface. WDYT? Separately though, **does anyone feel that backfill should still completely ignore pools by default?** Anyone think that behavior should be preserved? GitHub link: https://github.com/apache/airflow/discussions/43235 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
