> > Let me make sure I have the backwards compatibility straight.  If a user
> > switches to ORC 2.0, he could choose to continue writing in older formats
> > so that his old tools could read it
>    Yes, exactly.

To chime in on Owen's point, the development process has a slight wrinkle in 
it, which we avoided in the 0.11 -> 0.12 migration due to ORC being embedded in 

The feature addition is two-fold - the new features are available only when a 
user flips the writer versions.

There is no feature flag for reader versions, so the readers have to keep up to 
date with the writer changes (or just fail for the "blackholed" ones, with good 

Due to the split between projects, I expect to see a two-step development 
cycle, to clean up the integration pathways before the ABI is frozen in 2.0.

The entire process can be gated on the writer version - during the development 
process, there will be an experimental version (1.5?) and a stable version.

I have no interest in ever supporting an actual 1.5 version data setup in ORC, 
but for the sake of integration testing the 1.5->2.0 writer versions are 
extremely useful stepping stones towards a multi-project dependency like ORC.

Once the integrations are all complete and the format can be frozen, ORC 2.0 
releases can still disable the default writer version from being upgraded for 
another stable release.

After the ecosystem has had all its upgrades, the default version gets flipped 
to 2.0, while the ability to write 0.12 files will still remain as an option, 
while all intermediate reader versions will get dropped.

That's a bit more complicated than being part of Hive and sync'ing releases, 
but I think this gives ORC the flexibility to accept contributions from a wide 
community, supporting multi-project release timelines, without leaving the 
implementation full of reader implementations for many writer versions.


Reply via email to