potiuk commented on PR #54651: URL: https://github.com/apache/airflow/pull/54651#issuecomment-3227266973
Besides the falling tests, it looks cool. Scheme based messaging rather than figuring the implementation from the queue URL is a better approach for sure. I think we need to discuss and approach a deprecation mechanism for it. I would be for a rather "clean cut" with this one than a long deprecation. I see no particular reason to stick to the old mechanism if we convert all our message queue providers to the new versions. Those are still very early days for message queue, and i bet not many people have already built their own queues, and even if they did, switching to the new API is very straightforward. There are two options we could follow: 1) Sudden death 2) A little bit delayed death with exactly (1) version with deprecations (I don't think there is a point in making it a longer period) ## Sudden death What I would be for with this change is: * bump common.messaging to the new major version immediately (2.0.0) * instead of deprecation, fail when old discovery mechanism is used and explain how to convert your implementation * make all our providers that implement message queues depend on `common.messaging >= 2.0.0` Add "migration documentation" explaining how to migrate your implementation. There are a few drawback of it: * you will not be able to downgrad amazon, kafka, redis selectively - it's either all of them using new API (future versions of those providers), or all of them using old API (current versions) * old providers with new common.sql will fail. But this is only about common.messaging and if we explain in the error message a) what to do for custom providers b) need to upgrade for our providers - we should be good. ## Delayed (but still quick) death We could also, if we think above is too risky, do it in two steps: 1) Step 1. Common.messaging = 1.1.0 (with deprecation) + release all our providers with >=1.1.0 2) Step 2. (next provider wave): Common mesaging 2.0.0 (with failure as above) This will be a little more involved, but it has one advantage - you will be able to install common.messaging 1.1.0 and mix&match providers - some of them will use old and some new api. It does not seem like there is a "lot" of maintenance overhead for this - the only thing is that the implementation and releases will have to be spread over a month or so. I think it's worth to go with the deprecation pattern. But I would not delay it any further than till the following release. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
