hi Micah, Sorry for the delay in feedback. I looked at the document and it seems like a reasonable perspective about forward- and backward-compatibility.
It seems like the main thing you are proposing is to apply Semantic Versioning to Format and Library versions separately. That's an interesting idea, my thought had been to have a version number that is FORMAT_VERSION.LIBRARY_VERSION.PATCH_VERSION. But your proposal is more flexible in some ways, so let me clarify for others reading In what you are proposing, the next release would be: Format version: 1.0.0 Library version: 1.0.0 Suppose that 20 major versions down the road we stand at Format version: 1.5.0 Library version: 20.0.0 The minor version of the Format would indicate that there are additions, like new elements in the Type union, but otherwise backward and forward compatible. So the Minor version means "new things, but old clients will not be disrupted if those new things are not used". We've already been doing this since the V4 Format iteration but we have not had a way to signal that there may be new features. As a corollary to this, I wonder if we should create a dual version in the metadata PROTOCOL VERSION: (what is currently MetadataVersion, V2) FEATURE VERSION: not tracked at all So Minor version bumps in the format would trigger a bump in the FeatureVersion. Note that we don't really have a mechanism for clients and servers to report to each other what features they support, so this could help with that when for applications where it might matter. Should backward/forward compatibility be disrupted in the future, then a change to the major version would be required. So in year 2025, say, we might decide that we want to do: Format version: 2.0.0 Library version: 21.0.0 The Format version would live in the project's Documentation, so the Apache releases are only the library version. Regarding your open questions: 1. Should we clean up "warts" on the specification, like redundant information I don't think it's necessary. So if Metadata V5 is Format Version 1.0.0 (currently we are V4, but we're discussing some possible non-forward compatible changes...) I think that's OK. None of these things are "hurting" anything 2. Do we need additional mechanisms for marking some features as experimental? Not sure, but I think this can be mostly addressed through documentation. Flight will still be experimental in 1.0.0, for example. 3. Do we need protocol negotiation mechanisms in Flight Could you explain what you mean? Are you thinking if there is some major revamp of the protocol and you need to switch between a "V1 Flight Protocol" and a "V2 Flight Protocol"? - Wes On Thu, Jun 13, 2019 at 2:17 AM Micah Kornfield <emkornfi...@gmail.com> wrote: > > Hi Everyone, > I think there might be some ideas that we still need to reach consensus on > for how the format and libraries evolve in a post-1.0.0 release world. > Specifically, I think we need to agree on definitions for > backwards/forwards compatibility and its implications for versioning the > format. > > To this end I put some thoughts down in a Google Doc [1] for the purposes > of discussion. Comments welcome. I will start threads for any comments in > the document that seem to warrant further discussion, and once we reach > consensus I can create a patch to document what we decide on as part of the > specification. > > Thanks, > Micah > > [1] > https://docs.google.com/document/d/1uBitWu57rDu85tNHn0NwstAbrlYqor9dPFg_7QaE-nc/edit#
