I really think about a chapter (Which was missing): "How should I approach this migration?"
1) explain why there is no 1-1 migration instruction 2) explain that for every smart sensor they need to use or write deferrable operator 3) link to this information from "deprecation message" they will see in the logs when they use smart sensor (rather than relying on the fact that they will look at UPDATING.md and find the right part That's it, Guiding the users. Being helpful in finding the right information and context quickly (at the place where they hit the error and not in one of the 100 pages of documentation that they will only find by googling. J. On Thu, Dec 9, 2021 at 5:09 PM Kaxil Naik <[email protected]> wrote: > > Just as an FYI - the commit 18 hours ago on that PR already had added > "deprecation" in the docs too. > > Not only docs, but UPDATING.md, even in the Scheduler logs, so kudos to Jed > for taking care of it. > > So I don't agree with your comment or suggestion Jarek at least in the > context of this discussion as it makes me (at least) read that the PR does > not do those things. > > re: Tomek's question - it is a very valid question. Unfortunately, I don't > see a like-by-like replacement for DAG Authors as different work needs to be > done to write an Async operator and make a sensor "smart sensor compatible". > However, agree that we try to be as clear as possible on what a user might > need to do - I just don't know what that would be other than what I suggested > in last email and would love the feedback on the PR of what else can be > included. > > Thanks. > > Regards, > Kaxil > > On Thu, Dec 9, 2021 at 4:00 PM Kaxil Naik <[email protected]> wrote: >> >> I don't think there is a 1-1 migration path. Async operators supersede what >> Smart sensors were written to achieve - Cost Savings. >> >> Smart Sensors were marked experimental feature for the same reason and there >> are currently just two Sensors that are Smart >> sensors compatible. >> >> The only thing I can currently think of is writing an async version of the >> Smart Sensor Hook and Operator differs based on the underlying library that >> is used and >> https://airflow.apache.org/docs/apache-airflow/stable/concepts/deferring.html >> explains how you can write one. Also - >> https://airflow.apache.org/docs/apache-airflow/stable/concepts/deferring.html#smart-sensors >> >> >>> I believe we have done quite a bad job in the past assuming that our >>> users read all the discussions and AIPs we write. They don't. They >>> need some guidance. >> >> >> Which instances? I am just curious to know what are those bad instances >> where we "assumed" that our users read mailing list and not covered it in >> UPDATING.md or docs. >> >> Regards, >> Kaxil >> >> On Thu, Dec 9, 2021 at 3:46 PM Jarek Potiuk <[email protected]> wrote: >>> >>> Extremely good point Tomek. >>> >>> Also as Ephraim pointed out in the PR - IMHO any time when we do >>> deprecation we should have a note in our docs, explaining at the very >>> least how the users should approach the migration as correctly pointed >>> out by @turbaszek in the devlist. >>> I think this should be a standard of any deprecation we do. >>> >>> I believe we have done quite a bad job in the past assuming that our >>> users read all the discussions and AIPs we write. They don't. They >>> need some guidance. >>> >>> J. >>> >>> On Wed, Dec 8, 2021 at 11:44 PM Tomasz Urbaszek <[email protected]> >>> wrote: >>> > >>> > Do we have documentation about how to migrate from smart sensors to >>> > deferrable operators?
