Hi,

I have been working on the AIP-121 for the past few weeks which proposes
the introduction of an optional PollingTrigger class to airflow core to
standardize the lifecycle of polling-based triggers which I would like to
open up for a discussion.

The idea came from noticing that quite a few provider triggers
independently implement similar logic for polling intervals, retries,
backoff, timeouts, and terminal-state detection. During reviews where
triggers were being modified the same concerns were being raised with
inconsistent resolution. This naturally results in the different triggers
adopting their own lifecycle idiosyncrasies.

I think there's an opportunity to centralize these mechanics via a new base
class that these 'polling-based' triggers can be migrated to away from
BaseTrigger.

I have had some feedback from Shahar already on the AIP already which has
been implemented. I'd be interested to hear what others think, particularly
whether this abstraction is worthwhile or if smaller reusable helpers would
be sufficient. I'm also wondering whether this needs a formal AIP or could
be introduced through PRs and lazy consensus.

Please find below the link to the wiki page for the AIP:

https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/451974742/WIP+AIP-121+Introduce+a+PollingTrigger+Lifecycle+Framework

It will be moved to under discussion once I see some traction on here.

Thanks,
Sameer.

Reply via email to