potiuk commented on issue #27162: URL: https://github.com/apache/airflow/issues/27162#issuecomment-1296485077
I think this is far more than PR - it's more of an AIP. It's been raised by a number of people and there are similar discussions. For example we had a discussion https://github.com/apache/airflow/discussions/27325 about the very thing. Seems super useful, but IMHO this really needs a deliberate discussion and it needs not a "feature" but Airflow Improvement Proposal - https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals - we should discuss the approach, users, permissions, whether to make it plugs into current Sensors, Defferables, how to handle asyc/io approach, how we shoudl communicate between the "sensor" and webserver, how to handle cancellations, whether we should introduce separate state (this was proposed in https://github.com/apache/airflow/discussions/27325 . how it should be surfaced in the UI, etc. etc. I think at this moment there are more questions than answers and since it is crossing all layers in airflow, I think this is quite a big Proposal to make, not a feature. So rather than openinig a PR (or even opening an issue) I think if someone wants to implement it, starting with discussing it in devlist and writing an Airflow Improvement Proposal with detailed discussion about the design is a good idea (and it is very welcome - seems that this is highly requested feature). Maybe @dylanbstorey or @cdabella from the other thread would like to pick it up ? Or maybe someone else will soon ? Hopefully we will get it soon. For now - I am converting it into a discussion, it's far too big and not defined to be a feature. (but again it would be a welcome addition). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
