Yeah, It's nice. Shall I start it or wait for agreement about the names and templates?
Hossein Torabi blck...@gmail.com (mailto:blck...@gmail.com) On Apr 15 2021, at 2:53 pm, Sumit Maheshwari <sumeet.ma...@gmail.com> wrote: > Yeah, we've been using on_failure_callbacks for a long time and sending > alerts to slack and pagerduty. > > On Thu, Apr 15, 2021 at 4:30 PM Ash Berlin-Taylor <a...@apache.org > (mailto:a...@apache.org)> wrote: > > Yes, this sounds like a good feature, but I would go about it in a slightly > > different way that is less "special cased" to just chat services or > > webhooks and doesn't need new config (or much less) > > > > Specifically, rather than having to add new parameters to dag/task for > > webhooks specifically, I would suggest we have add utility functions to > > hooks/operators etc that have the behaviour you specify. > > > > Slack/credential configuration should be done using connections, not in the > > config file > > > > So something like this becomes the dag/task API: > > > > from airflow.providers.slack.hooks import task_callback_handler as > > send_slack_message > > > > task = MyOperator( > > task_id = "something", > > on_failure_callback= send_slack_message( > > slack_conn_id='slack-default', # Default, not actually required here, for > > example only > > > > channels=['#data-ops'], > > mentions=['@ash'], > > ), > > ) > > > > Names can be changed/discussed, this is just a rough idea. > > > > Using this approach means that a) We don't need to add a whole bunch of new > > config, and b) it can be easily extended/created by providers without > > needing changes to core -- after all the feature to run code after > > failure/success already exists, we just want to "package up" the common > > task of sending a slack message. > > > > I think for consistency of interface too we should deprecate the > > email_on_failure task attribute too in favour of a similar function. > > > > And finally, perhaps `on_*_callback` gets extended to allow a list of > > functions instead of just one. > > > > Thoughts? > > > > -ash > > > > > > On Thu, 15 Apr, 2021 at 12:42, Sayed Mohammad Hossein Torabi > > <blck...@gmail.com (mailto:blck...@gmail.com)> wrote: > > > I would like to present a new feature in airflow. It's a good idea to > > > support webhooks triggers for notifying users about their tasks. > > > we can do it by providing it in a way that I could supports software like > > > Slack, MatterMost, Discord, MS Teams. this software currently uses by a > > > wide range of international users. > > > This feature provides a better UX for our users and also reduces efforts > > > for every company/user who uses airflow to write their own callbacks. > > > > > > Here is my initials idea: > > > Changes in the default_args: > > > add new entry: webhook_on_failure: bool > > > > > > add new entry: webhook_on_retry: bool > > > > > > add new entry: webhook_on_success: bool > > > > > > add new entry: channels: List[str] (List of channels) > > > > > > add new entry: mentions: List[str] (List of users to mention) > > > > > > > > > > > > Changes in the airflow.cfg: > > > add new seciont like [webhook] for Slack, etc. configuration. > > > > > > > > > > > > Message templates: > > > I would like to do it like the Prometheus alerting template. > > > > > > > > > About message templates, I mean something like this. > > > > > >
