On Thu, Apr 23, 2015 at 4:13 PM, Stack <st...@duboce.net> wrote: > Does this 'admission' help with the which-hadoop thread too? >
Right -- "working toward semver". Are we now at liberty to bump to 2.6 or 2.7 even for branch-1.1? I still have Karthik's offer to roll a 2.5.3 with the HDFS issue resolved. What about the jackson issue with 2.5 YARN runtime? Thanks Sean and Josh for being our community conscience on these issues. > Just want to make sure we're all on the same page (or find out if not). > > > > On Thu, Apr 23, 2015 at 2:59 PM, Enis Söztutar <e...@apache.org> wrote: > > > > > Then let's get Andrew's proposal committed: > > > > > > + APIs available in a patch version will be available in all later > > > + patch versions. However, new APIs may be added which will not be > > > + available in earlier patch versions. > > > > > > I propose the following change to the "Client Binary compatibility" > > section > > > of Section 11.1: > > > > > > - Old client code can run unchanged (no recompilation needed) against > new > > > jars. > > > + Client code written to APIs available in a given patch release > > > + can run unchanged (no recompilation needed) against the new > > > + jars of later patch versions. > > > > > > We can even claim that if you are not using new APIs, we are binary > > compat > > > for downgrade. But you have to make sure that your code compiles with > > that > > > downgraded version. > > > > > > Enis > > > > > > > > > > > > On Thu, Apr 23, 2015 at 2:04 PM, Sean Busbey <bus...@cloudera.com> > > wrote: > > > > > > > I'm fine with us adding methods to public APIs in patch releases so > > long > > > as > > > > we stop claiming to follow semver. We can say we take the principles > of > > > > semver as inspiration. This would reflect the current state of the > > world > > > > WRT 1.0.1 and would still give us a reason keep the narrower > definition > > > of > > > > "new features" in minor versions. > > > > > > > > > > > > No longer claiming semver would also have the added benefit of making > > it > > > > for me to easier to explain our compatibility promises to people. > Right > > > now > > > > I have to explain the difference between the things that get proper > > > semver > > > > treatment (e.g. public api, wire compat) and which things are > > downgraded > > > to > > > > breaking-on-minor (e.g. LimitedPrivate, binary compatibility). > Starting > > > > with the context of "we like semver" will be easier than "we use > > semver". > > > > > > > > > > > > Like Josh, my main concern is that we accurately advertise what we're > > > > doing. There are few things I've found more frustrating than being an > > end > > > > user of a project that claims to follow semver without understanding > > the > > > > burden that carries (and subsequently not meeting it). > > > > > > > > On Thu, Apr 23, 2015 at 3:48 PM, Stack <st...@duboce.net> wrote: > > > > > > > > > I agree w/ the Enis characterization (so we need the callout on > > semvar) > > > > but > > > > > think we should practice what Seans says (patch is bug fixes only). > > > > > St.Ack > > > > > > > > > > On Thu, Apr 23, 2015 at 1:31 PM, Sean Busbey <bus...@cloudera.com> > > > > wrote: > > > > > > > > > > > Why don't we just focus development after a minor release on the > > next > > > > > minor > > > > > > release instead of the next patch release? > > > > > > > > > > > > We could limit backports to the patch releases to critical bugs, > > > which > > > > > > would cut down on how often someone has to deal with the pain of > > > making > > > > > > sure we don't add to public APIs. It also reduces the risk > someone > > > > going > > > > > > through an upgrade has, since there are fewer changes. > > > > > > > > > > > > If someone fixes a bug and doesn't want to do the work of making > > sure > > > > it > > > > > > doesn't add methods in a patch release, they just don't backport > to > > > > that > > > > > > version and make a follow on e.g. "backport to 1.0.z" ticket. > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Apr 23, 2015 at 1:50 PM, Enis Söztutar < > enis....@gmail.com > > > > > > > > wrote: > > > > > > > > > > > > > +1 to the proposal. > > > > > > > > > > > > > > The problem is that we have a very big API surface especially > > with > > > > the > > > > > > > coprocessors included in the report. Even simple bug fixes can > > > > > introduce > > > > > > > protected or public methods to base classes, which makes patch > > > > releases > > > > > > > very hard to maintain. I would not want to spend the effort to > > > spend > > > > > tons > > > > > > > of time trying to make a patch not introduce new methods in > order > > > to > > > > > > > backport. That effort can be spent elsewhere IMO. > > > > > > > > > > > > > > Looking at the report > > > > > > > > > https://people.apache.org/~enis/1.0.0_1.0.1RC2_compat_report.html, > > > > > > nothing > > > > > > > strikes me as "new functionality". Going from current 1.0.0 to > > > > 1.0.1RC2 > > > > > > > should actually be as you would expect from upgrading a patch > > > > release. > > > > > > > > > > > > > > Yes, adding new API in patch releases will make downgrading > > harder, > > > > > but I > > > > > > > think that is an acceptable tradeoff. We can document that if > > your > > > > > > > application compiles (meaning that you are not using new API) > > with > > > > > 1.0.0, > > > > > > > then you can swap your jars in a binary compat manner. > > > > > > > > > > > > > > Enis > > > > > > > > > > > > > > On Thu, Apr 23, 2015 at 10:03 AM, Andrew Purtell < > > > > apurt...@apache.org> > > > > > > > wrote: > > > > > > > > > > > > > > > Anyone disagree with the point of view put forward by Josh > and > > > > Sean? > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Apr 22, 2015 at 10:35 AM, Josh Elser < > > > josh.el...@gmail.com > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Andy -- I understood your intent, but thanks for > clarifying. > > > (as > > > > > well > > > > > > > as > > > > > > > > > taking the time to break this discussion out in the first > > > > place). I > > > > > > > agree > > > > > > > > > with your assessment. > > > > > > > > > > > > > > > > > > re: Sean's comments, if it wasn't clear by me asking in the > > > first > > > > > > > place, > > > > > > > > I > > > > > > > > > also think sticking as close as possible to semver's rules > is > > > the > > > > > > best > > > > > > > > > approach, although I'm getting the impression that there > have > > > > been > > > > > > some > > > > > > > > > previous reservations to doing so (especially by your > comment > > > > about > > > > > > > > > backporting features if there is demand is). > > > > > > > > > > > > > > > > > > I've found adhering to the bug-fix release restrictions can > > be > > > a > > > > > very > > > > > > > > > painful and time-consuming task, so this is something to > get > > a > > > > > > > > > representative sampling of those who do the work to make > sure > > > > > > everyone > > > > > > > is > > > > > > > > > on board. > > > > > > > > > > > > > > > > > > > > > > > > > > > Sean Busbey wrote: > > > > > > > > > > > > > > > > > >> I'd much rather we stick with the definitions used in > > Semantic > > > > > > > > Versioning. > > > > > > > > >> Our use is already confusing enough given our matrix of > > > > > > > compatibilities > > > > > > > > >> that don't get "major version for breaking" protections. > > > > > > > > >> > > > > > > > > >> We've previously discussed how we'll do additional minor > > > > releases > > > > > > when > > > > > > > > >> there's sufficient interest in the new features present > > there. > > > > > > What's > > > > > > > > >> building that demand if any backwards compatible change > can > > go > > > > > back > > > > > > > > into a > > > > > > > > >> patch release? > > > > > > > > >> > > > > > > > > >> Would we have an easier time restraining ourselves if we > > had a > > > > > > regular > > > > > > > > >> schedule planned around new minor versions? > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> On Wed, Apr 22, 2015 at 12:03 PM, Josh Elser< > > > > josh.el...@gmail.com > > > > > > > > > > > > > > >> wrote: > > > > > > > > >> > > > > > > > > >> While I can understand the desire to want to add things, > I > > do > > > > > think > > > > > > > it > > > > > > > > >>> makes things harder for users to reliably write code > > against > > > > > > versions > > > > > > > > of > > > > > > > > >>> HBase which (by their view) should be completely > compatible > > > > with > > > > > > one > > > > > > > > >>> another. > > > > > > > > >>> > > > > > > > > >>> Take this extremely hypothetical situation: I'm new to > > HBase > > > > and > > > > > > > start > > > > > > > > >>> writing some code against HBase 1.0.1 which was just > > deployed > > > > at > > > > > my > > > > > > > > >>> $job. I > > > > > > > > >>> don't _know_ what APIs are new, I just know what exists > and > > > > treat > > > > > > > that > > > > > > > > as > > > > > > > > >>> acceptable for me to be using. Meanwhile in production, > > some > > > > > other > > > > > > > > people > > > > > > > > >>> find a bug with HBase 1.0.1 and roll back to 1.0.0 which > > they > > > > had > > > > > > > been > > > > > > > > >>> previously using. My reaction would be "of course my code > > > > should > > > > > > work > > > > > > > > >>> with > > > > > > > > >>> HBase 1.0.0, I only used the public API" when in fact > this > > is > > > > not > > > > > > > true. > > > > > > > > >>> > > > > > > > > >>> Personally, I think it's a little bold to say semver is > > even > > > in > > > > > use > > > > > > > if > > > > > > > > >>> this principal isn't being followed as it doesn't follow > at > > > all > > > > > > with > > > > > > > my > > > > > > > > >>> understanding on the guarantees defined by semver for > > bug-fix > > > > > > > releases. > > > > > > > > >>> > > > > > > > > >>> That being said, if the intent *is* to allow ourselves to > > > make > > > > > > these > > > > > > > > >>> sorts > > > > > > > > >>> of changes, I just think some sort of disclaimer should > be > > > > > present: > > > > > > > > >>> > > > > > > > > >>> - HBase uses Semantic Versioning for its release > versioning > > > > > > > > >>> + HBase uses Semantic Versioning for its release > versioning > > > > with > > > > > a > > > > > > > > caveat > > > > > > > > >>> that methods and members might be added in newer bug-fix > > > > releases > > > > > > > that > > > > > > > > >>> were > > > > > > > > >>> not present in the previous bug-fix release. > > > > > > > > >>> > > > > > > > > >>> > > > > > > > > >>> Andrew Purtell wrote: > > > > > > > > >>> > > > > > > > > >>> [Subject changed] > > > > > > > > >>>> > > > > > > > > >>>> On Tue, Apr 21, 2015 at 8:47 PM, Josh Elser< > > > > > josh.el...@gmail.com> > > > > > > > > >>>> wrote: > > > > > > > > >>>> > > > > > > > > >>>> I was a little surprised when I noticed method > additions > > > to > > > > > > > > >>>> > > > > > > > > >>>>> InterfaceAudience.Public annotated classes. This means > > > that a > > > > > > user > > > > > > > > >>>>> could > > > > > > > > >>>>> write code against 1.0.1 that would not work against > > 1.0.0 > > > > > which > > > > > > > > seems > > > > > > > > >>>>> undesirable for a bugfix release. I read over the book > > > > section > > > > > on > > > > > > > > >>>>> compatibility and didn't see this addressed, so I > thought > > > I'd > > > > > > ask. > > > > > > > > >>>>> > > > > > > > > >>>>> > > > > > > > > >>>> Let's clarify this. It's not the first time this > question > > > has > > > > > been > > > > > > > > >>>> asked. > > > > > > > > >>>> > > > > > > > > >>>> To get things moving: > > > > > > > > >>>> > > > > > > > > >>>> I propose the following addition to the "Client API > > > > > compatibility" > > > > > > > > >>>> section > > > > > > > > >>>> of Section 11.1: > > > > > > > > >>>> > > > > > > > > >>>> + APIs available in a patch version will be available in > > all > > > > > later > > > > > > > > >>>> + patch versions. However, new APIs may be added which > > will > > > > not > > > > > be > > > > > > > > >>>> + available in earlier patch versions. > > > > > > > > >>>> > > > > > > > > >>>> I propose the following change to the "Client Binary > > > > > > compatibility" > > > > > > > > >>>> section > > > > > > > > >>>> of Section 11.1: > > > > > > > > >>>> > > > > > > > > >>>> - Old client code can run unchanged (no recompilation > > > needed) > > > > > > > against > > > > > > > > >>>> new > > > > > > > > >>>> jars. > > > > > > > > >>>> + Client code written to APIs available in a given patch > > > > release > > > > > > > > >>>> + can run unchanged (no recompilation needed) against > the > > > new > > > > > > > > >>>> + jars of later patch versions. > > > > > > > > >>>> > > > > > > > > >>>> > > > > > > > > >>>> What do you think? > > > > > > > > >>>> > > > > > > > > >>>> If these changes are (mostly) ok, then this clarifies in > > one > > > > > > > > direction. > > > > > > > > >>>> > > > > > > > > >>>> If these changes are not acceptable, I will propose > edits > > > that > > > > > > > clarify > > > > > > > > >>>> toward the opposite meaning. > > > > > > > > >>>> > > > > > >
