And I agree with you :) (but with a twist). I do not say we should remove "UPDATING.md" information. Not at all.
Providing that: * UPDATING.md contains both * It is available in our User-facing docs * it has an anchor to this particular "piece of upgrading" * the deprecated error message has a direct link to it to help to find it I (and our users I hope) would be perfectly happy. As far as I know. UPDATING.md is only in Github (I just checked and I could not find in in airflow.apache.org. So by definition it's not a User documentation. It's developer documentation only. J On Thu, Dec 9, 2021 at 5:20 PM Kaxil Naik <[email protected]> wrote: > > Partially agree -- not completely. > > Firstly what I agree - (1) and (2) points from your email. > > Disagree the (3) point and the para after that. > > UPDATING.md is our source of breaking changes. Instead of users just having > to rely and checking "deprecation" for 100s of commands, we should be helpful > to users by also having a single page where we list all the deprecations. > > That is another way of being helpful in finding the "right" information and > context quickly too. And "Guiding the users" in a different way. > > On Thu, Dec 9, 2021 at 4:13 PM Jarek Potiuk <[email protected]> wrote: >> >> I really think about a chapter (Which was missing): >> >> "How should I approach this migration?" >> >> 1) explain why there is no 1-1 migration instruction >> 2) explain that for every smart sensor they need to use or write >> deferrable operator >> 3) link to this information from "deprecation message" they will see >> in the logs when they use smart sensor (rather than relying on the >> fact that they will look at UPDATING.md and find the right part >> >> That's it, Guiding the users. Being helpful in finding the right >> information and context quickly (at the place where they hit the error >> and not in one of the 100 pages of documentation that they will only >> find by googling. >> >> J. >> >> On Thu, Dec 9, 2021 at 5:09 PM Kaxil Naik <[email protected]> wrote: >> > >> > Just as an FYI - the commit 18 hours ago on that PR already had added >> > "deprecation" in the docs too. >> > >> > Not only docs, but UPDATING.md, even in the Scheduler logs, so kudos to >> > Jed for taking care of it. >> > >> > So I don't agree with your comment or suggestion Jarek at least in the >> > context of this discussion as it makes me (at least) read that the PR does >> > not do those things. >> > >> > re: Tomek's question - it is a very valid question. Unfortunately, I don't >> > see a like-by-like replacement for DAG Authors as different work needs to >> > be done to write an Async operator and make a sensor "smart sensor >> > compatible". >> > However, agree that we try to be as clear as possible on what a user might >> > need to do - I just don't know what that would be other than what I >> > suggested in last email and would love the feedback on the PR of what else >> > can be included. >> > >> > Thanks. >> > >> > Regards, >> > Kaxil >> > >> > On Thu, Dec 9, 2021 at 4:00 PM Kaxil Naik <[email protected]> wrote: >> >> >> >> I don't think there is a 1-1 migration path. Async operators supersede >> >> what Smart sensors were written to achieve - Cost Savings. >> >> >> >> Smart Sensors were marked experimental feature for the same reason and >> >> there are currently just two Sensors that are Smart >> >> sensors compatible. >> >> >> >> The only thing I can currently think of is writing an async version of >> >> the Smart Sensor Hook and Operator differs based on the underlying >> >> library that is used and >> >> https://airflow.apache.org/docs/apache-airflow/stable/concepts/deferring.html >> >> explains how you can write one. Also - >> >> https://airflow.apache.org/docs/apache-airflow/stable/concepts/deferring.html#smart-sensors >> >> >> >> >> >>> I believe we have done quite a bad job in the past assuming that our >> >>> users read all the discussions and AIPs we write. They don't. They >> >>> need some guidance. >> >> >> >> >> >> Which instances? I am just curious to know what are those bad instances >> >> where we "assumed" that our users read mailing list and not covered it in >> >> UPDATING.md or docs. >> >> >> >> Regards, >> >> Kaxil >> >> >> >> On Thu, Dec 9, 2021 at 3:46 PM Jarek Potiuk <[email protected]> wrote: >> >>> >> >>> Extremely good point Tomek. >> >>> >> >>> Also as Ephraim pointed out in the PR - IMHO any time when we do >> >>> deprecation we should have a note in our docs, explaining at the very >> >>> least how the users should approach the migration as correctly pointed >> >>> out by @turbaszek in the devlist. >> >>> I think this should be a standard of any deprecation we do. >> >>> >> >>> I believe we have done quite a bad job in the past assuming that our >> >>> users read all the discussions and AIPs we write. They don't. They >> >>> need some guidance. >> >>> >> >>> J. >> >>> >> >>> On Wed, Dec 8, 2021 at 11:44 PM Tomasz Urbaszek <[email protected]> >> >>> wrote: >> >>> > >> >>> > Do we have documentation about how to migrate from smart sensors to >> >>> > deferrable operators?
