As the cutoff for 3.1 approaches, it has become pretty clear that the full scope of work for Deadline Alerts won't be done in time to meet the 3.1 code freeze. We can launch right now with support for both custom async callbacks and async Notifiers (see caveats below), but sync callbacks are not going to be ready for the deadline. Ramit will be sending out another message shortly which addresses the issues around synchronous callbacks and how we can bridge them, but it has turned into a non-trivial project unto itself which has some overlap with the proposed AIP-92 [1], which has a conflicting approach to solving the same issues. In the meantime, I think we will need to focus on shipping async-only support in 3.1.
Async Notifiers live in their respective provider packages, which is a great benefit here. Their release cycles are shorter and we can push them out as they are completed, but some are going to require some domain knowledge. Here's the caveat: there are currently zero published Notifiers which fully support async operation. We have a PR up [2] which updates the SlackWebhookNotifier, I have another coming for the other SlackNotifier, and Ramit will have one shortly for the HttpNotifier. So here's my call for help. Using those three examples, it would be great to get as many Notifiers updated as possible. Most are pretty easy, some will be complicated. I have not gone through them all as we were REALLY trying to focus on getting sync callbacks working and this was intended to be presented as a community mob-coding project after we got the rest of the Deadlines code done. We really want to get email, slack, and http done for launch; the rest can follow as folks get the time or inclination. [1] https://cwiki.apache.org/confluence/display/AIRFLOW/%5BWIP%5D+AIP-92+Isolate+DAG+processor%2C+Callback+processor%2C+and+Triggerer+from+core+services [2] https://github.com/apache/airflow/pull/53831 <https://cwiki.apache.org/confluence/display/AIRFLOW/%5BWIP%5D+AIP-92+Isolate+DAG+processor%2C+Callback+processor%2C+and+Triggerer+from+core+services> - ferruzzi
