My understanding is that we are not adding the client portion to the KIP is because we believe it will require a lot of discussion (read: will be really hard to get right).
Maybe it is a hint that this protocol is too difficult for clients to implement? I can't see why is it easy in C and Python and super difficult in Java. Even if it is too hard to implement just in Java for some reason (god knows the language has issues), isn't it a good reason to come up with something we can implement in a reasonable way? Gwen On Fri, Apr 1, 2016 at 10:58 AM, Ashish Singh <asi...@cloudera.com> wrote: > That is a fair concern and I think eventually we might want to have java > clients backwards compatible. However, blocking KIP-35 on that might not be > the best idea. The reason I say so is due to following reasons. > > 1. Backwards compatibility in java clients is a larger discussion, as we > have already seen. Having a separate KIP focussing exactly on that will > help in reducing moving pieces in one KIP. > 2. It probably will fall out of 0.10 due to tight timeline, I am assuming > we do not want to delay 0.10 a lot. > 3. Even if we make java clients backward compatible in 0.10, I do not think > it will be able to work with older releases as older broker versions still > won't provide info on supported api versions. If we add api versions > req/resp in 0.10, and add backwards compatibility in java clients in later > versions, they will probably work with 0.10 and we will be able to test > that. > > On Fri, Apr 1, 2016 at 10:28 AM, Ewen Cheslack-Postava <e...@confluent.io> > wrote: > > > On Fri, Apr 1, 2016 at 10:22 AM, Gwen Shapira <g...@confluent.io> wrote: > > > > > I have two concerns with the proposal as it is: > > > > > > 1. Having an API that publishes protocol is useless for clients if > > > developers don't bump the API when they should. > > > I would like to see good documentation on when protocols are bumped > and a > > > proposal on how this will be automatically validated to the extent > > possible > > > (and to what extent are we still at risk of accidental breakage). > > > Even if we don't implement all of this at first pass, I want to know > > which > > > direction we are going in order to solve the client compatibility > issue. > > > > > > 2. In addition to third-party clients, there are stream processing > > > frameworks who use the Java client we publish as their client and would > > > also like to enjoy the same compatibility benefits C and Python clients > > > will enjoy. It will be very silly if Apache Kafka clients are the worst > > > clients out there from compatibility POV. The KIP can be implemented in > > > parts, but I really want to see an effort to build the Java client > > > compatibility into the KIP and if possible into the release too. > > > > > > > I wouldn't conflate those two things. Changing the compatibility approach > > of the Java clients could easily be another KIP (and probably a large > one, > > too). There are already high quality, well maintained clients trying to > use > > a different approach and this addresses their needs. Blocking the entire > > ecosystem on the Java client seems problematic. > > > > That said, agreed that it would be bad for the Java client to have the > > worst compatibility story. > > > > -Ewen > > > > > > > > > > Gwen > > > > > > On Thu, Mar 31, 2016 at 2:51 PM, Jason Gustafson <ja...@confluent.io> > > > wrote: > > > > > > > Bumping this thread. I talked with Ashish and Magnus about this KIP > > > offline > > > > and I'm gradually coming over. The new API actually stands by itself > > > > outside of the discussion about whether the client should support > > > backwards > > > > compatibility or not. For the Java client, we could continue to > support > > > the > > > > current compatibility approach in which the client supports only > > brokers > > > > with the same version or greater. In that case, we would use this API > > > only > > > > to assert that the current API versions are all supported, and raise > an > > > > exception if they are not. This gives us the capability going forward > > to > > > > detect when the client is talking to an older broker, which we don't > > have > > > > right now. This check should be straightforward, so we could do it > now, > > > > which would resolve some of the uneasiness about having an unused > > feature > > > > which we depended on other clients to test for us. Does that make > sense > > > or > > > > not? > > > > > > > > -Jason > > > > > > > > On Thu, Mar 17, 2016 at 4:06 PM, Ashish Singh <asi...@cloudera.com> > > > wrote: > > > > > > > > > We have proposed and discussed majorly three approaches so far, > there > > > > were > > > > > many minor versions with small variations. Comparing them really > > > > requires a > > > > > side by side proposal and their pros/cons, and I agree with others > > that > > > > > this has been lacking in the KIP. We just updated the KIP with > > > following > > > > > details. > > > > > > > > > > 1. Provide proposed changes in all the three proposals we have > > > discussed > > > > so > > > > > far. Except the current proposal, these proposals are in rejected > > > > > alternatives. > > > > > 2. Provide reasoning on why the rejected proposals were rejected. > > > > > 3. Add scenarios for all of these proposals from a client developer > > and > > > > > core Kafka developer point of view. > > > > > > > > > > As we are really close to 0.10 deadline, a quick round of voting > will > > > > > really help. If you really do not like the idea, please feel free > to > > > say > > > > > so. If the vote fails for the current proposal, it can at lease > > provide > > > > > recommendations that we should consider for next version of > proposal > > > and > > > > > put it up for vote again for next release. However, as stated > earlier > > > by > > > > > multiple people having this ASAP will be awesome. > > > > > > > > > > On Thu, Mar 17, 2016 at 3:29 PM, Dana Powers < > dana.pow...@gmail.com> > > > > > wrote: > > > > > > > > > > > On Thu, Mar 17, 2016 at 1:42 PM, Gwen Shapira <g...@confluent.io > > > > > > wrote: > > > > > > > > > > > > > "I think I would make this approach work by looking at the > > released > > > > > > server > > > > > > > version documentation for each version that I am trying to > > support > > > > and > > > > > > test > > > > > > > against*, manually identify the expected "protocol vectors" > each > > > > > > supports, > > > > > > > store that as a map of vectors to "broker versions", check each > > > > vector > > > > > at > > > > > > > runtime until I find a match, and write code compatibility > checks > > > > from > > > > > > > there." > > > > > > > > > > > > > > How is this better than a global version ID? > > > > > > > > > > > > > > > > > > As a client developer, it seems roughly the same. I think it > > probably > > > > > > avoids the server development workflow issues, and possibly the > > need > > > to > > > > > > agree on semantics of the global version ID? But others surely > are > > > more > > > > > > qualified than me to comment on that part. > > > > > > > > > > > > -Dana > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Regards, > > > > > Ashish > > > > > > > > > > > > > > > > > > > > -- > > Thanks, > > Ewen > > > > > > -- > > Regards, > Ashish >
