GitHub user potiuk added a comment to the discussion: Dynamic Pool Allocation Based on Runtime Parameters
Currently not possible because pools are used as selection criteria when you select which tasks to schedule. If you would like to invest your time and effort in figuring out what to change, you would have to dig deep in how scheduler works and probably figure another mechanism that could be used more dynamically, but this is rather serious architectural change and would require somoene (you or maybe you can find and encourage someone to attempt it) - to write an AIP (Airflow Improvement Proposal) - https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals and likely a POC (Proof of Concept implementing it). This is open-source and it means that whoever wants might attempt it and contribute it. GitHub link: https://github.com/apache/airflow/discussions/47860#discussioncomment-12526336 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
