Great news!!! And thanks Gabor and Xinli for handling the release process! Cheers, Gidon
On Thu, Mar 25, 2021 at 7:01 PM Xinli shang <[email protected]> wrote: > Thanks everybody for the verification and special thanks to all the > contributors to this release! This release includes awesome features and > improvements. We look forward to the industry's adoption! > > On Thu, Mar 25, 2021 at 3:35 AM Gabor Szadovszky <[email protected]> wrote: > > > The vote to release 1.12.0 RC4 as Apache Parquet MR 1.12.0 is PASSED with > > the required three +1 binding votes and two +1 non-binding votes. (There > > were no -1 or 0 votes.) > > Thank you all who verified and voted! > > > > I'm going forward with the release process soon. > > > > On Thu, Mar 25, 2021 at 1:26 AM Julien Le Dem <[email protected]> > wrote: > > > > > +1 (binding) > > > I verified the signature and built from source. > > > All tests pass. > > > Looks good. > > > > > > On Wed, Mar 24, 2021 at 2:07 AM Gabor Szadovszky <[email protected]> > > wrote: > > > > > > > I currently have the feeling that the Avro/Jackson related issue has > > been > > > > discussed and the community agrees on moving forward with this RC as > is > > > > (without upgrading the Avro and the Jackson dependencies). > > > > So, I'm giving my +1 (binding) vote. > > > > > > > > On Tue, Mar 23, 2021 at 9:28 PM Aaron Niskode-Dossett > > > > <[email protected]> wrote: > > > > > > > > > +1 (non-binding) > > > > > > > > > > - cloned the 1.12.0-rc-4 tag from github > > > > > - compiled jars locally and all tests passed > > > > > - used the 1.12.0 jars as dependencies for a local application that > > > > streams > > > > > data into protobuf-parquet files > > > > > - confirmed data is correct and can be read with parquet-tools > > compiled > > > > > from parquet 1.11.1 > > > > > > > > > > On Tue, Mar 23, 2021 at 10:47 AM Xinli shang > <[email protected] > > > > > > > > wrote: > > > > > > > > > > > Let's discuss it in today's community sync meeting. > > > > > > > > > > > > On Tue, Mar 23, 2021 at 8:37 AM Aaron Niskode-Dossett > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > Gabor and Ismaël, thank you both for the very clear > explanations > > of > > > > > > what's > > > > > > > going on. > > > > > > > > > > > > > > Based on Gabor's description of avro compatibility I would be > +1 > > > > > > > (non-binding) for the current RC. > > > > > > > > > > > > > > On Tue, Mar 23, 2021 at 4:36 AM Gabor Szadovszky < > > [email protected] > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Thanks, Ismaël for the explanation. I have a couple of notes > > > about > > > > > your > > > > > > > > concerns. > > > > > > > > > > > > > > > > - Parquet 1.12.0 as per the semantic versioning is not a > major > > > but > > > > a > > > > > > > minor > > > > > > > > release. (It is different from the Avro versioning strategy > > where > > > > the > > > > > > > > second version number means major version changes.) > > > > > > > > - The jackson dependency is shaded in the parquet jars so the > > > > > > > > synchronization of the version is not needed (and not even > > > > possible). > > > > > > > > - Using the latest Avro version makes sense but if we do not > > use > > > it > > > > > for > > > > > > > the > > > > > > > > current release it should not cause any issues in our > clients. > > > > Let's > > > > > > > check > > > > > > > > the following example. We upgrade to the latest 1.10.2 Avro > > > release > > > > > in > > > > > > > > parquet then release it under 1.12.0. Later on Avro creates a > > new > > > > > > release > > > > > > > > (e.g. 1.10.3 or even 1.11.0) while Parquet does not. In this > > case > > > > our > > > > > > > > clients need to upgrade Avro without Parquet. If it is a > major > > > Avro > > > > > > > release > > > > > > > > it might occur that the Parquet code has to be updated but > > > usually > > > > it > > > > > > is > > > > > > > > not the case. (The last time we've had to change production > > code > > > > for > > > > > an > > > > > > > > Avro upgrade was from 1.7.6 to 1.8.0.) I think our clients > > should > > > > be > > > > > > able > > > > > > > > to upgrade Avro independently from Parquet and vice versa > > (until > > > > > there > > > > > > > are > > > > > > > > incompatibility issues). I would even change Parquet's Avro > > > > > dependency > > > > > > to > > > > > > > > "provided" but that might be a breaking change and clearly > > won't > > > do > > > > > it > > > > > > > just > > > > > > > > before the release. > > > > > > > > > > > > > > > > What do you think? Anyone have a strong opinion about this > > topic? > > > > > > > > > > > > > > > > Cheers, > > > > > > > > Gabor > > > > > > > > > > > > > > > > On Mon, Mar 22, 2021 at 6:31 PM Ismaël Mejía < > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > > > > > Sure. The Avro upgrade feature/API wise is minor for > Parquet, > > > so > > > > > the > > > > > > > > > possibility of adding a regression is really REALLY minor. > > The > > > > > hidden > > > > > > > > issue > > > > > > > > > is the new transitive dependencies introduced by Avro, > > > concretely > > > > > > > Jackson > > > > > > > > > 2.12.2. > > > > > > > > > > > > > > > > > > Since Parquet 1.12.0 is a major version it is probably a > good > > > > > moment > > > > > > to > > > > > > > > > upgrade Jackson too that's why I opened [1] (already > merged). > > > In > > > > > > > > particular > > > > > > > > > now that Spark merged support for both Avro 1.10.2 [1] and > > > > Jackson > > > > > > > 2.12.2 > > > > > > > > > [2] for the upcoming 3.2.0 release, so now Spark can easily > > > bring > > > > > > > > upgraded > > > > > > > > > Parquet too with all the dependencies well aligned. This of > > > > course > > > > > is > > > > > > > > not a > > > > > > > > > blocker for the release or for other downstream projects > but > > it > > > > > might > > > > > > > > help > > > > > > > > > to make their life better because they will have less > > > dependency > > > > > > > > alignment > > > > > > > > > issues to battle. > > > > > > > > > > > > > > > > > > Ismaël > > > > > > > > > > > > > > > > > > [1] https://github.com/apache/parquet-mr/pull/883 > > > > > > > > > [2] https://github.com/apache/spark/pull/31866 > > > > > > > > > [3] https://github.com/apache/spark/pull/31878 > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Mar 22, 2021, 3:37 PM Xinli shang > > > > <[email protected] > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hi Ismaël, > > > > > > > > > > > > > > > > > > > > Can you explain a little bit more on if we don't upgrade > in > > > > this > > > > > > > > release, > > > > > > > > > > what could be the worst-case scenario for the ecosystem? > > The > > > > > > > > last-minute > > > > > > > > > > upgrading seems a rush to me but I would like to hear > what > > > are > > > > > the > > > > > > > > impact > > > > > > > > > > if we don't. As Gabor mentioned, this should not be a > > > > > > show-stopper. > > > > > > > > > > > > > > > > > > > > Xinli > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Mar 22, 2021 at 1:55 AM Gabor Szadovszky < > > > > > [email protected] > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Thanks a lot for the ones who checked the release and > > > voted. > > > > We > > > > > > > have > > > > > > > > > two > > > > > > > > > > +1 > > > > > > > > > > > votes (1 binding) so far. > > > > > > > > > > > > > > > > > > > > > > About the Avro + Jackson upgrade. I would like to hear > > your > > > > > > > opinions > > > > > > > > > > about > > > > > > > > > > > the current vote. Do you think we shall fail the > current > > > vote > > > > > to > > > > > > > > > upgrade > > > > > > > > > > > these versions for the next RC or you accept this RC as > > is > > > > for > > > > > > the > > > > > > > > next > > > > > > > > > > > release. (NOTE: as per semantic versioning this release > > in > > > > not > > > > > a > > > > > > > > major > > > > > > > > > > but > > > > > > > > > > > a minor release of parquet-mr.) > > > > > > > > > > > > > > > > > > > > > > I give a +0 (binding) for now until we have a clear > view > > > > about > > > > > > the > > > > > > > > > > previous > > > > > > > > > > > topic. > > > > > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > > Gabor > > > > > > > > > > > > > > > > > > > > > > On Sat, Mar 20, 2021 at 12:44 PM Grisha Weintraub < > > > > > > > > > > > [email protected]> wrote: > > > > > > > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > > > > > > > Verified hash, signature and successful build. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Mar 17, 2021 at 11:22 AM Gabor Szadovszky < > > > > > > > > [email protected]> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > > > > > > > > > > > > > > > I propose the following RC to be released as the > > > official > > > > > > > Apache > > > > > > > > > > > Parquet > > > > > > > > > > > > > 1.12.0 release. > > > > > > > > > > > > > > > > > > > > > > > > > > The commit id is > > > db75a6815f2ba1d1ee89d1a90aeb296f1f3a8f20 > > > > > > > > > > > > > * This corresponds to the tag: > > > apache-parquet-1.12.0-rc4 > > > > > > > > > > > > > * > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/parquet-mr/tree/db75a6815f2ba1d1ee89d1a90aeb296f1f3a8f20 > > > > > > > > > > > > > > > > > > > > > > > > > > The release tarball, signature, and checksums are > > here: > > > > > > > > > > > > > * > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/parquet/apache-parquet-1.12.0-rc4 > > > > > > > > > > > > > > > > > > > > > > > > > > You can find the KEYS file here: > > > > > > > > > > > > > * https://downloads.apache.org/parquet/KEYS > > > > > > > > > > > > > > > > > > > > > > > > > > Binary artifacts are staged in Nexus here: > > > > > > > > > > > > > * > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://repository.apache.org/content/groups/staging/org/apache/parquet/ > > > > > > > > > > > > > > > > > > > > > > > > > > This release includes the features Parquet Modular > > > > > Encryption > > > > > > > and > > > > > > > > > > > Parquet > > > > > > > > > > > > > Bloom Filter. See details at: > > > > > > > > > > > > > * > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/parquet-mr/blob/apache-parquet-1.12.0-rc4/CHANGES.md > > > > > > > > > > > > > > > > > > > > > > > > > > Please download, verify, and test. > > > > > > > > > > > > > > > > > > > > > > > > > > Please vote in the next 72 hours. > > > > > > > > > > > > > > > > > > > > > > > > > > [ ] +1 Release this as Apache Parquet 1.12.0 > > > > > > > > > > > > > [ ] +0 > > > > > > > > > > > > > [ ] -1 Do not release this because... > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Xinli Shang > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Aaron Niskode-Dossett, Data Engineering -- Etsy > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Xinli Shang > > > > > > > > > > > > > > > > > > > > > -- > > > > > Aaron Niskode-Dossett, Data Engineering -- Etsy > > > > > > > > > > > > > > > > > -- > Xinli Shang >
