Hi, As a short update, I just checked the PR for PARQUET-1419 and although in its current form it is a breaking change, it can be easily rewritten to become backwards-compatible so this part of the problem does not apply any more.
Br, Zoltan On Tue, Sep 25, 2018 at 7:10 PM Zoltan Ivanfi <[email protected]> wrote: > Hi, > > On the Parquet sync we discussed that the practice of maintaining a copy > of parquet.thrift in parquet-cpp is dangerous and that we must take care to > not release parquet-format changes in parquet-cpp before we officially > release them in parquet-format. As I got back to my computer and started to > create a JIRA about this, I noticed that unfortunately this has already > happened. > > The encryption-releated parquet.thrift changes have not only been added to > only parquet-format, but to parquet-cpp as well, and these changes got > released in parquet-cpp 1.5.0. This is very unfortunate, because > PARQUET-1419 would change the encryption in a breaking way, which is only > acceptable as long as the original is not released. Additionally, it has > been discussed that a formal voting should take place before incorporating > the encryption features in the format. > > Now that parquet-cpp has already shipped these changes, we must choose the > lesser evil of the following two options: > > - Release a parquet-cpp 1.6.0 with a breaking change and risk that > encrypted data files already written with parquet-cpp 1.5.0 will not be > readable any more. > - Release the encryption in parquet-format as it is, regardless of > voting results and discard PARQUET-1419. > > Personally I have a hard time deciding which one I consider lesser evil. > What are your opinions? > > Thanks, > > Zoltan >
