+1 (binding) In <CAJPUwMCsMG5pga5h=+slr97ogkk52ecr8ti-9oftwlhvgf1...@mail.gmail.com> "[VOTE] Increment MetadataVersion in Schema.fbs from V4 to V5 for 1.0.0 release" on Mon, 29 Jun 2020 16:42:45 -0500, Wes McKinney <wesmck...@gmail.com> wrote:
> Hi, > > As discussed on the mailing list [1], in order to demarcate the > pre-1.0.0 and post-1.0.0 worlds, and to allow the > forward-compatibility-protection changes we are making to actually > work (i.e. so that libraries can recognize that they have received > data with a feature that they do not support), I have proposed to > increment the MetadataVersion from V4 to V5. Additionally, if the > union validity bitmap changes are accepted, the MetadataVersion could > be used to control whether unions are permitted to be serialized or > not (with V4 -- used by v0.8.0 to v0.17.1, unions would not be > permitted). > > Since there have been no backward incompatible changes to the Arrow > format since 0.8.0, this would be no different, and (aside from the > union issue) libraries supporting V5 are expected to accept BOTH V4 > and V5 so that backward compatibility is not broken, and any > serialized data from prior versions of the Arrow libraries (0.8.0 > onward) will continue to be readable. > > Implementations are recommended, but not required, to provide an > optional "V4 compatibility mode" for forward compatibility > (serializing data from >= 1.0.0 that needs to be readable by older > libraries, e.g. Spark deployments stuck on an older Java-Arrow > version). In this compatibility mode, non-forward-compatible features > added in 1.0.0 and beyond would not be permitted. > > A PR with the changes to Schema.fbs (possibly subject to some > clarifying changes to the comments) is at [2]. > > Once the PR is merged, it will be necessary for implementations to be > updated and tested as appropriate at minimum to validate that backward > compatibility is preserved (i.e. V4 IPC payloads are still readable -- > we have some in apache/arrow-testing and can add more as needed). > > The vote will be open for at least 72 hours. > > [ ] +1 Accept addition of MetadataVersion::V5 along with its general > implications above > [ ] +0 > [ ] -1 Do not accept because... > > [1]: > https://lists.apache.org/thread.html/r856822cc366d944b3ecdf32c2ea9b1ad8fc9d12507baa2f2840a64b6%40%3Cdev.arrow.apache.org%3E > [2]: https://github.com/apache/arrow/pull/7566
