As for UPDATING only being on github, I have a separate proposal in that area coming soon. It likely won't be an issue come time to release 2.3.0 👍.
On Thu, Dec 9, 2021 at 9:34 AM Jarek Potiuk <[email protected]> wrote: > And I agree with you :) (but with a twist). > > I do not say we should remove "UPDATING.md" information. Not at all. > > Providing that: > > * UPDATING.md contains both > * It is available in our User-facing docs > * it has an anchor to this particular "piece of upgrading" > * the deprecated error message has a direct link to it to help to find it > > I (and our users I hope) would be perfectly happy. > > As far as I know. UPDATING.md is only in Github (I just checked and I > could not find in in airflow.apache.org. So by definition it's not a > User documentation. It's developer documentation only. > > J > > > > > On Thu, Dec 9, 2021 at 5:20 PM Kaxil Naik <[email protected]> wrote: > > > > Partially agree -- not completely. > > > > Firstly what I agree - (1) and (2) points from your email. > > > > Disagree the (3) point and the para after that. > > > > UPDATING.md is our source of breaking changes. Instead of users just > having to rely and checking "deprecation" for 100s of commands, we should > be helpful to users by also having a single page where we list all the > deprecations. > > > > That is another way of being helpful in finding the "right" information > and context quickly too. And "Guiding the users" in a different way. > > > > On Thu, Dec 9, 2021 at 4:13 PM Jarek Potiuk <[email protected]> wrote: > >> > >> I really think about a chapter (Which was missing): > >> > >> "How should I approach this migration?" > >> > >> 1) explain why there is no 1-1 migration instruction > >> 2) explain that for every smart sensor they need to use or write > >> deferrable operator > >> 3) link to this information from "deprecation message" they will see > >> in the logs when they use smart sensor (rather than relying on the > >> fact that they will look at UPDATING.md and find the right part > >> > >> That's it, Guiding the users. Being helpful in finding the right > >> information and context quickly (at the place where they hit the error > >> and not in one of the 100 pages of documentation that they will only > >> find by googling. > >> > >> J. > >> > >> On Thu, Dec 9, 2021 at 5:09 PM Kaxil Naik <[email protected]> wrote: > >> > > >> > Just as an FYI - the commit 18 hours ago on that PR already had added > "deprecation" in the docs too. > >> > > >> > Not only docs, but UPDATING.md, even in the Scheduler logs, so kudos > to Jed for taking care of it. > >> > > >> > So I don't agree with your comment or suggestion Jarek at least in > the context of this discussion as it makes me (at least) read that the PR > does not do those things. > >> > > >> > re: Tomek's question - it is a very valid question. Unfortunately, I > don't see a like-by-like replacement for DAG Authors as different work > needs to be done to write an Async operator and make a sensor "smart sensor > compatible". > >> > However, agree that we try to be as clear as possible on what a user > might need to do - I just don't know what that would be other than what I > suggested in last email and would love the feedback on the PR of what else > can be included. > >> > > >> > Thanks. > >> > > >> > Regards, > >> > Kaxil > >> > > >> > On Thu, Dec 9, 2021 at 4:00 PM Kaxil Naik <[email protected]> > wrote: > >> >> > >> >> I don't think there is a 1-1 migration path. Async operators > supersede what Smart sensors were written to achieve - Cost Savings. > >> >> > >> >> Smart Sensors were marked experimental feature for the same reason > and there are currently just two Sensors that are Smart > >> >> sensors compatible. > >> >> > >> >> The only thing I can currently think of is writing an async version > of the Smart Sensor Hook and Operator differs based on the underlying > library that is used and > https://airflow.apache.org/docs/apache-airflow/stable/concepts/deferring.html > explains how you can write one. Also - > https://airflow.apache.org/docs/apache-airflow/stable/concepts/deferring.html#smart-sensors > >> >> > >> >> > >> >>> I believe we have done quite a bad job in the past assuming that our > >> >>> users read all the discussions and AIPs we write. They don't. They > >> >>> need some guidance. > >> >> > >> >> > >> >> Which instances? I am just curious to know what are those bad > instances where we "assumed" that our users read mailing list and not > covered it in UPDATING.md or docs. > >> >> > >> >> Regards, > >> >> Kaxil > >> >> > >> >> On Thu, Dec 9, 2021 at 3:46 PM Jarek Potiuk <[email protected]> > wrote: > >> >>> > >> >>> Extremely good point Tomek. > >> >>> > >> >>> Also as Ephraim pointed out in the PR - IMHO any time when we do > >> >>> deprecation we should have a note in our docs, explaining at the > very > >> >>> least how the users should approach the migration as correctly > pointed > >> >>> out by @turbaszek in the devlist. > >> >>> I think this should be a standard of any deprecation we do. > >> >>> > >> >>> I believe we have done quite a bad job in the past assuming that our > >> >>> users read all the discussions and AIPs we write. They don't. They > >> >>> need some guidance. > >> >>> > >> >>> J. > >> >>> > >> >>> On Wed, Dec 8, 2021 at 11:44 PM Tomasz Urbaszek < > [email protected]> wrote: > >> >>> > > >> >>> > Do we have documentation about how to migrate from smart sensors > to deferrable operators? >
