I don't think deferrable can act as a replacement at this point. Removing/changing defaults of poke and reschedule in favor of deferrable means that deferrable becomes a critical feature. This means that any bug related to it must be triaged and fixed ASAP. We are not there yet. I don't think many maintainers know this area of the code.
In my perspective as a user I have a functionality that is working fine. I learned to trust it. I know how it behaves and am able to write code efficiently with it. The suggested replacement is a feature that has 30ish bug reports with little community focus. That is a lot of risk with small profit to gain. For me this is an indication of -1 to this proposal. For example issue https://github.com/apache/airflow/issues/36734 with suggested PR: https://github.com/apache/airflow/pull/33718 I admit I am not into the details but if we accept this proposal this bug becomes top priority problem and this is open since 2023. So my position is -1 for any change about poke and reschedule (even for changing defaults). On Thu, Nov 14, 2024 at 10:28 PM Jarek Potiuk <ja...@potiuk.com> wrote: > > I think that if we have implemented async for an operator, we should > remove > the non-async version. > > I think we all agree here. The question is only "when". I see two options: > > 1) When Airflow 3 is released > 2) When Airflow 2 stops being supported > > I'd be for 2) . > > Why? Because it has the least impact on our users while it does not make > our life much harder, while we might commit to clean-up eventually. > > I personally (with my ADHD personality twist) often have an urge to fix > things "now". It's very rewarding and has great "feedback" of feeling > better. But I think the next best thing is to have a clear timeline to > remove things like that and relentlessly follow it through. And it has the > added benefits of avoiding surprises for users who follow our decisions and > policies. > > I think the biggest problem I have with things that are deprecated are not > that they are lying around, but with the feeling that they will stay there > forever (aka - we will not clean our technical debt). I think having a > policy when we remove things and following that, avoids that problem (as > long as we do follow). > > See for example this PR for Google Provider that Max opened today: > https://github.com/apache/airflow/pull/43953.= ~ -8000 lines of code we > will not have to maintain. and whoever followed it, they had 5 months or so > to adjust as Google team set some rules for that and in most cases set > the deprecation date. This is cool. Most of those warnings have a > "planned_removal_date" set. > > And I think it also shows our maturity - that we do not "jump" on changing > something, but we plan it, we have a reason behind it, whe know why we are > doing it. what are consequences to our users, and follow through, giving > enough warning to the users (those who care to listen). > > And the more we do it, the more we plan and announce in advance and > relentlessly follow it through, the more our users will listen to those > things. > > J. > > > On Thu, Nov 14, 2024 at 4:12 PM Daniel Standish > <daniel.stand...@astronomer.io.invalid> wrote: > > > To me, I don't really mind reschedule mode so much. But I *really *don't > > like that we have *both* styles implemented in operators (talking about > > providers here). > > > > I think that if we have implemented async for an operator, we should > remove > > the non-async version. > > > > And we should remove the `deferrable` param for 3.0 -- this is something > > that to me makes no sense, since it implies that sensors should have both > > ways implemented. > > > > But users should still be allowed to write rescheduling sensors. > > >
