That's fair enough too. Guozhang
On Fri, Jul 21, 2017 at 12:13 PM, Ismael Juma <isma...@gmail.com> wrote: > Yes, I agree that the choice of version number can be done separately. > That's why I said I'd file a separate JIRA for the documentation > improvements. Having said that, there are some expectations that people > have for projects that have reached 1.0.0 and we should try to allocate > time for the important ones. > > Ismael > > On 21 Jul 2017 8:07 pm, "Guozhang Wang" <wangg...@gmail.com> wrote: > > > Thanks Ismael. I agree with you on all these points, and for some of > these > > points like 3) we never have a written-down policy though in practice we > > tend to follow some patterns. > > > > To me deciding what's the version number of the next major release does > not > > necessarily mean we need now to change any of these or to set the hard > > rules along with it; I'd like to keep them as two separate discussions as > > they seem semi-orthogonal to me. > > > > > > Guozhang > > > > > > On Fri, Jul 21, 2017 at 8:44 AM, Ismael Juma <ism...@juma.me.uk> wrote: > > > > > On the topic of documentation, we should also document which releases > are > > > still supported and which are not. There a few factors to consider: > > > > > > 1. Which branches receive bug fixes. We typically backport fixes to the > > two > > > most recent stable branches (the most recent stable branch typically > gets > > > more backports than the older one). > > > > > > 2. Which branches receive security fixes. This could be the same as > `1`, > > > but we could attempt to backport more aggressively for security fixes > as > > > they tend to be rare (so far at least) and the impact could be severe. > > > > > > 3. The release policy for stable branches. We tend to stop releasing > > from a > > > given stable branch before we stop backporting fixes. Maybe that's OK, > > but > > > it would be good to document how we decide that a bug fix release is > > > needed. > > > > > > 4. How long are direct upgrades supported for. During the time-based > > > releases discussion, we agreed to support direct upgrades for 2 years. > As > > > it happens, direct upgrades from 0.8.2 to 1.0.0 would not be supported > if > > > we follow this strictly. Not clear if we want to do that. > > > > > > 5. How long are older clients supported for. Current brokers support > > > clients all the way to 0.8.x. > > > > > > 6. How long are older brokers supported for. Current clients support > > 0.10.x > > > and newer brokers. > > > > > > 7. How long are message formats supported for. We never discussed > this. I > > > think 5 years would probably be the minimum. > > > > > > Ismael > > > > > > P.S. I'll file a JIRA to capture this information. > > > > > > On Wed, Jul 19, 2017 at 10:12 AM, Ismael Juma <ism...@juma.me.uk> > wrote: > > > > > > > Hi Stevo, > > > > > > > > Thanks for your feedback. We should definitely do a better job of > > > > documenting things. We basically follow semantic versioning, but it's > > > > currently a bit confusing because: > > > > > > > > 1. There are 4 segments in the version. The "0." part should be > ignored > > > > when deciding what is major, minor and patch at the moment, but many > > > people > > > > don't know this. Once we move to 1.0.0, that problem goes away. > > > > > > > > 2. To know what is a public API, you must check the Javadoc ( > > > > https://kafka.apache.org/0110/javadoc/index.html?org/ > > > > apache/kafka/clients/consumer/KafkaConsumer.html). If it's not > listed > > > > there, it's not public API. Ideally, it would be obvious from the > > package > > > > name (i.e. there would be "internals" in the name), but we are not > > there > > > > yet. The exception are the old Scala APIs, but they have all been > > > > deprecated and they will be removed eventually (the old Scala > consumers > > > > won't be removed until the June 2018 release at the earliest in order > > to > > > > give people time to migrate). > > > > > > > > 3. Even though we are following reasonably common practices, we > haven't > > > > documented them all in one place. It would be great to do it during > the > > > > next release cycle. > > > > > > > > A few comments below. > > > > > > > > On Wed, Jul 19, 2017 at 1:31 AM, Stevo Slavić <ssla...@gmail.com> > > wrote: > > > > > > > >> - APIs not labeled or labeled as stable > > > >> -- change in major version is only one that can break backward > > > >> compatibility (client APIs or behavior) > > > >> > > > > > > > > To clarify, stable APIs should not be changed in an incompatible way > > > > without a deprecation cycle. Independently of whether it's a major > > > release > > > > or not. > > > > > > > > > > > >> -- change in minor version can introduce new features, but not break > > > >> backward compatibility > > > >> -- change in patch version, is for bug fixes only. > > > >> > > > > > > > > Right, this has been the case for a while already. Also see > annotations > > > > below. > > > > > > > > > > > >> - APIs labeled as evolving can be broken in backward incompatible > way > > in > > > >> any release, but are assumed less likely to be broken compared to > > > unstable > > > >> APIs > > > >> - APIs labeled as unstable can be broken in backward incompatible > way > > in > > > >> any release, major, minor or patch > > > >> > > > > > > > > The relevant annotations do explain this: > > > > > > > > https://kafka.apache.org/0110/javadoc/org/apache/kafka/ > > > common/annotation/ > > > > InterfaceStability.html > > > > https://kafka.apache.org/0110/javadoc/org/apache/kafka/ > > > common/annotation/ > > > > InterfaceStability.Stable.html > > > > https://kafka.apache.org/0110/javadoc/org/apache/kafka/ > > > common/annotation/ > > > > InterfaceStability.Evolving.html > > > > https://kafka.apache.org/0110/javadoc/org/apache/kafka/ > > > common/annotation/ > > > > InterfaceStability.Unstable.html > > > > > > > > But we should have a section in our documentation as well. > > > > > > > > > > > >> - deprecated stable APIs are treated as any stable APIs, they can be > > > >> removed only in major release, are not allowed to be changed in > > backward > > > >> incompatible way in either patch or minor version release > > > >> > > > > > > > > Right, but note that stable non-deprecated APIs provide stronger > > > > guarantees in major releases (they can't be changed in an > incompatible > > > way). > > > > > > > >> > > > >> This means one should be able to upgrade server and recompile/deploy > > > apps > > > >> with clients to new minor.patch release with dependency version > change > > > >> being only change needed and there would be no drama. > > > >> > > > > > > > > That should have been the case for a while as long as you are using > > > stable > > > > public APIs. > > > > > > > >> > > > >> Practice/"features" like protocol version being a parameter, and > > > >> defaulting > > > >> to latest so auto updated with dependency update which introduces > new > > > >> protocol/behavior should not be used in public client APIs. To > switch > > > >> between backward incompatible APIs (contract and behaviors), ideally > > > user > > > >> should explicitly have to change code and not dependency only, but > at > > > >> least > > > >> it should be clearly communicated that there are breaking changes to > > > >> expect > > > >> even with just dependency update by e.g. giving major version > release > > > >> clear > > > >> meaning. If app dependency on Kafka client library minor.patch on > same > > > >> major is updated, and if there's a change in behavior or API > requiring > > > app > > > >> code change - it's a bug. > > > >> > > > > > > > > Hmm, if the protocol bump provides improved behaviour, that is not a > > > > backwards incompatible change though. So, I don't think I agree with > > > this. > > > > Of course, > > > > it does mean that _downgrading_ may cause loss of functionality. > That's > > > > OK, in my opinion. > > > > > > > > Change introduced contrary to the SLO, is OK to be reported as bug. > > > >> Everything else is improvement or feature request. > > > >> > > > >> If this was the case, and 1.0.0 was released today with APIs as they > > are > > > >> now, Scala client APIs even though deprecated would not break and > > > require > > > >> refactoring with every 1.* minor/patch release, and would only be > > > allowed > > > >> to be broken or removed in future major release, like 2.0.0 > > > >> > > > > > > > > Yes, that is the plan for any _public_ Scala client APIs that are > still > > > > present in 1.0.0. The public Scala client APIs are the producer and > > > > consumer, basically. Again, we should make this clear in our > > > documentation. > > > > Note that we have made an effort to keep those APIs compatible for > > quite > > > a > > > > while. It sounds like you have had some issues, were they related to > > > usage > > > > of internal Admin APIs by any chance (since we didn't have a public > > > > AdminClient API until very recently)? > > > > > > > >> > > > >> It should be also clear how long is each version supported - e.g. if > > > >> minor.patch had meaning that there are no backward incompatible > > changes, > > > >> it's OK to file a bug only for current major.minor.patch; previous > > major > > > >> and its last minor.patch can only have patches released up to some > > time > > > >> like 1 up to 3 months. > > > >> > > > > > > > > I am not sure I understood this point correctly. Can you please > > clarify? > > > > > > > > If there are changes in release cadence with new versioning, it > should > > be > > > >> clear too. > > > >> > > > > > > > > No changes are planned. We have started time-based releases less > than a > > > > year ago and they seem to be going well. > > > > > > > > Ismael > > > > > > > > > > > > > > > -- > > -- Guozhang > > > -- -- Guozhang
