Just to be sure I fully understand the proposal: For the Library Version, we are going to increment the MAJOR version on every normal release, and increment the MINOR version if we need to release a patch/bug fix type of release.
Since SemVer allows for API breaking changes on MAJOR versions, this basically means, each library (C++, Python, C#, Java, etc) _can_ introduce API breaking changes on every normal release (like we have been with the 0.x.0 releases). So, for example, we release library v1.0.0 in a few months and then library v2.0.0 a few months after that. In v2.0.0, C++, Python, and Java didn't make any breaking API changes from 1.0.0. But C# made 3 API breaking changes. This would be acceptable? If my understanding above is correct, then I think this is a good plan. Initially I was concerned that the C# library wouldn't be free to make API breaking changes with making the version `1.0.0`. The C# library is still pretty inadequate, and I have a feeling there are a few things that will need to change about it in the future. But with the above plan, this concern won't be a problem. Eric -----Original Message----- From: Micah Kornfield <emkornfi...@gmail.com> Sent: Monday, July 1, 2019 10:02 PM To: Wes McKinney <wesmck...@gmail.com> Cc: dev@arrow.apache.org Subject: Re: [Discuss] Compatibility Guarantees and Versioning Post "1.0.0" Hi Wes, Thanks for your response. In regards to the protocol negotiation your description of feature reporting (snipped below) is along the lines of what I was thinking. It might not be necessary for 1.0.0, but at some point might become useful. > Note that we don't really have a mechanism for clients and servers to > report to each other what features they support, so this could help > with that when for applications where it might matter. Thanks, Micah On Mon, Jul 1, 2019 at 12:54 PM Wes McKinney <wesmck...@gmail.com> wrote: > hi Micah, > > Sorry for the delay in feedback. I looked at the document and it seems > like a reasonable perspective about forward- and > backward-compatibility. > > It seems like the main thing you are proposing is to apply Semantic > Versioning to Format and Library versions separately. That's an > interesting idea, my thought had been to have a version number that is > FORMAT_VERSION.LIBRARY_VERSION.PATCH_VERSION. But your proposal is > more flexible in some ways, so let me clarify for others reading > > In what you are proposing, the next release would be: > > Format version: 1.0.0 > Library version: 1.0.0 > > Suppose that 20 major versions down the road we stand at > > Format version: 1.5.0 > Library version: 20.0.0 > > The minor version of the Format would indicate that there are > additions, like new elements in the Type union, but otherwise backward > and forward compatible. So the Minor version means "new things, but > old clients will not be disrupted if those new things are not used". > We've already been doing this since the V4 Format iteration but we > have not had a way to signal that there may be new features. As a > corollary to this, I wonder if we should create a dual version in the > metadata > > PROTOCOL VERSION: (what is currently MetadataVersion, V2) FEATURE > VERSION: not tracked at all > > So Minor version bumps in the format would trigger a bump in the > FeatureVersion. Note that we don't really have a mechanism for clients > and servers to report to each other what features they support, so > this could help with that when for applications where it might matter. > > Should backward/forward compatibility be disrupted in the future, then > a change to the major version would be required. So in year 2025, say, > we might decide that we want to do: > > Format version: 2.0.0 > Library version: 21.0.0 > > The Format version would live in the project's Documentation, so the > Apache releases are only the library version. > > Regarding your open questions: > > 1. Should we clean up "warts" on the specification, like redundant > information > > I don't think it's necessary. So if Metadata V5 is Format Version > 1.0.0 (currently we are V4, but we're discussing some possible > non-forward compatible changes...) I think that's OK. None of these > things are "hurting" anything > > 2. Do we need additional mechanisms for marking some features as > experimental? > > Not sure, but I think this can be mostly addressed through > documentation. Flight will still be experimental in 1.0.0, for > example. > > 3. Do we need protocol negotiation mechanisms in Flight > > Could you explain what you mean? Are you thinking if there is some > major revamp of the protocol and you need to switch between a "V1 > Flight Protocol" and a "V2 Flight Protocol"? > > - Wes > > On Thu, Jun 13, 2019 at 2:17 AM Micah Kornfield > <emkornfi...@gmail.com> > wrote: > > > > Hi Everyone, > > I think there might be some ideas that we still need to reach > > consensus > on > > for how the format and libraries evolve in a post-1.0.0 release world. > > Specifically, I think we need to agree on definitions for > > backwards/forwards compatibility and its implications for versioning > > the format. > > > > To this end I put some thoughts down in a Google Doc [1] for the > > purposes of discussion. Comments welcome. I will start threads for > > any comments > in > > the document that seem to warrant further discussion, and once we > > reach consensus I can create a patch to document what we decide on > > as part of > the > > specification. > > > > Thanks, > > Micah > > > > [1] > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs > .google.com%2Fdocument%2Fd%2F1uBitWu57rDu85tNHn0NwstAbrlYqor9dPFg_7QaE-nc%2Fedit%23&data=02%7C01%7CEric.Erhardt%40microsoft.com%7C6fc59049ffb049c9ddb108d6fe99bebf%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636976334243577292&sdata=YNQ%2FgL5rvmvRqvvW%2Bxjmb%2F4KeEe2JHe1ruws2VP%2BvK4%3D&reserved=0 >
