Thanks all. I am happy that we are making progress to close this out.

As mentioned in my (long) email last Friday, I also think that the KIP makes a lot of implicit assumptions, and it might benefit from begin more explicit, even explaining things that are "set in stone" already, and the KIP does not change them (like API deprecation timelines and guarantees). But as Bruno said, it would help a lot to make it easier to understand the KIP.

Only a very small number of people seems to fully understand all the nuances, and it took quite a while to collect and curate all the cases, to shape this KIP... And we know that a lot of people actually read KIPs, and the title for KIP-1124 is or sure click-bait...

I am also happy to edit the wiki page directly if that's ok with Kuan, to add background information that I believe is missing and important.



My overall understanding of the state of the KIP, ie, its actual content seems to be, that there is a high level agreement, and the content of the KIP is ok.

The only open question seems to be, if we want to recommend `2.8.x` as a bridge release for Kafka Streams or not. As said in my email on Friday, I am personally ok either way, with a preference to add `2.8.x` as bridge release, as it simplifies updating the code by avoiding compilation errors. With a 2.8.x bridge release, KS application code would _always_ compile, and the JavaDocs help a lot to guide users how to migrate off deprecated APIs. -- With a direct upgrade from older versions to 3.9.x, code does not compile, and there is no guidance as the old methods (and corresponding JavaDocs) are already removed, making it significant harder for user to update their code and make it compile again.

In the end, we would only _recommend_ 2.8.x as a bridge release to users, and there won't be any claim that an direct upgrade to 3.9.x is not supported. If one wants to do this, it's tested, and ok. Knock yourself out -- there is for sure apps out there, that are just a simple `builder.stream(...).filter(...).to(...)` the there won't be an API incompatibility issues.

But I don't think it would be wise to _recommend_ a direct upgrade to 3.9.x for the reasons stated, as many apps are more complex.

Thoughts?



About testing: I did look into the system tests, and it seems we actually don't really have a real upgrade test for clients, ie, a test that is running an consumer or producer with some older versions, and executes a rolling bounce to upgrade the app...

But maybe such a test is not necessary for clients (compare to Kafka Streams), and the existing client-broker forward/backward compatibility test are good enough? (At least in the last 10 years, it seems this test coverage was sufficient?)

I want to point out though that I agree with Bruno: If we assume that the client-broker compatibility tests are sufficient to verify client upgrades, we do effectively/implicitly test upgrading 2.0 (and older clients) to 4.0 using 3.9 and older brokers. These test coverage does not exist in a single branch so, but test from different branches combined, implicitly cover this. Nevertheless, is it tested.

So it seems both Bruno and Ismael are right, depending how you look at the problem. In any case, even if we test such an upgrade implicitly, I agree that it's good to exclude these upgrades officially as unsupported.



-Matthias




On 2/27/25 6:10 AM, Ismael Juma wrote:
Hi Bruno,

A comment below.

On Thu, Feb 27, 2025, 1:20 AM Bruno Cadonna <cado...@apache.org> wrote:

BC2:
Another aspect that confuses me in the KIP is the bridge version for
clients. If my broker version is 3.9.x, I can do a direct upgrade from
1.0.x to 4.0.x (let apart the Java API compatibility mentioned above).
The 4.0.x release tests 4.0.x clients against 3.9.x brokers and the
3.9.x release tests 1.0.x clients against 3.9.x brokers. It is not clear
to me why we need a bridge release in this case.


We don't test client upgrades from 1.0.x to 4.0.x. They may work, but it's
untested. Again, we are focusing on simplicity versus flexibility for edge
cases. The number of people upgrading from 1.0 to 4.0 is very low (if not
zero).

That's why I still think the simple description I provided in my other
message is the way to go

Deprecated APIs may or may not affect users, so that has to be evaluated on
a case by case basis.

Ismael


Reply via email to