Good point Jim, Personally, I'm also looking forward to the next release of Apache Parquet format.
I took the liberty of creating an umbrella ticket to get an overview of the blockers that we want to get in the 2.7 release. The ticket: https://jira.apache.org/jira/browse/PARQUET-1608 Regarding your question, I don't have an opinion on 1, but I think 2 is very important. In the end, the parquet format is nothing more than a couple of Thrift definitions. I would suggest writing good unit tests to ensure that the bloom filters behave in the same manner. The release process can be done by a committer, but also requires the involvement of at least 3 PMC's [1] to get the binding votes to get the release signed off. Cheers, Fokko [1] https://www.apache.org/foundation/voting.html Op do 20 jun. 2019 om 02:00 schreef Jim Apple <[email protected]>: > This is a thread for discussing a release of parquet-format. The last > release appears to be 2.6.0 from September 2018: > > https://github.com/apache/parquet-format/releases > > The diff from then until now is > > > https://github.com/apache/parquet-format/compare/df6132b94f273521a418a74442085fdd5a0aa009...4157b4c6132086e318943f1898523f7dcb013f35 > > It's my understanding we'll need a parquet-format release before a > parquet-mr and/or parquet-cpp release[0]. > > In the most recent discussion thread on this, there were two concerns > raised that are not yet addressed: > > 1. Should Bloom filters use a different hash function by default? > 2. Should we devise an automated way to test cross-language compatibility > of parquet files, especially for the new Bloom filter spec? > > I am suggesting a release despite these open issues based on my belief > that it's reasonable to handle these after a parquet-format release. > > A final note: although I am suggesting the release, it looks to me like > the release recipe[1] can only be executed by a committer, which I am not. > This means even if there is consensus on a release, someone else would need > to do the legwork. > > Thanks, > Jim > > [0] This code now lives in > https://github.com/apache/arrow/tree/master/cpp/src/parquet, I believe. > > [1] https://parquet.apache.org/documentation/how-to-release/ >
