potiuk edited a comment on issue #9755: URL: https://github.com/apache/airflow/issues/9755#issuecomment-695791077
> I created a PR with a cluster policy that assigns pool based on connection: https://github.com/apache/airflow/pull/11033/files I left some comments - I think the problem with this workaround is that without support from the framework, it would not work well for things like transfer operators where two hooks are used with different connection ids. So I think we should at least describe it as a 'half-baked" solution that does not work in all cases. What I think this issue is about is to try to make sure that there is always a relation "task" <> connections no matter how it is defined and that we can "drain" the task by draining the connection. I am afraid this is a bit more than just a simple policy to add in a more "general" case. I think currently we simply lack this relation in Airflow and I think (for the lineage usage in the future), it is a nice thing to add. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
