> > Let me make sure I have the backwards compatibility straight. If a user
> > switches to ORC 2.0, he could choose to continue writing in older formats
> > so that his old tools could read it
> Yes, exactly.
To chime in on Owen's point, the development process has a slight wrinkle in
it, which we avoided in the 0.11 -> 0.12 migration due to ORC being embedded in
The feature addition is two-fold - the new features are available only when a
user flips the writer versions.
There is no feature flag for reader versions, so the readers have to keep up to
date with the writer changes (or just fail for the "blackholed" ones, with good
Due to the split between projects, I expect to see a two-step development
cycle, to clean up the integration pathways before the ABI is frozen in 2.0.
The entire process can be gated on the writer version - during the development
process, there will be an experimental version (1.5?) and a stable version.
I have no interest in ever supporting an actual 1.5 version data setup in ORC,
but for the sake of integration testing the 1.5->2.0 writer versions are
extremely useful stepping stones towards a multi-project dependency like ORC.
Once the integrations are all complete and the format can be frozen, ORC 2.0
releases can still disable the default writer version from being upgraded for
another stable release.
After the ecosystem has had all its upgrades, the default version gets flipped
to 2.0, while the ability to write 0.12 files will still remain as an option,
while all intermediate reader versions will get dropped.
That's a bit more complicated than being part of Hive and sync'ing releases,
but I think this gives ORC the flexibility to accept contributions from a wide
community, supporting multi-project release timelines, without leaving the
implementation full of reader implementations for many writer versions.