Thanks Fokko, It seems then that getting Avro 1.9.1 is a better idea, but in that case the question is still the same, can this be backported on 1.11.x, eventually for 1.11.1 ? I ask this because after looking at Parquet's release pace I am worried that we will have to wait to long to get it in.
On Mon, Jun 3, 2019 at 3:26 PM Driesprong, Fokko <[email protected]> wrote: > > Ismaël, > > From what I understand from Jacob Tolar, I believe we have a blocking issue > in Apache Avro: https://issues.apache.org/jira/browse/AVRO-2400 We would > need to release 1.9.1 first to get this sorted out. > > Regarding Parquet 1.11.0, I would expect a branch which has been cut from > master. However, I don't see this branch on Github. > > Cheers, Fokko > > Op ma 3 jun. 2019 om 15:19 schreef Ismaël Mejía <[email protected]>: > > > Any chance the upgrade to Avro 1.9.0 may be cherry picked (otherwise > > it will take longer to get this downstreamed into Spark). > > > > I can work on the cherry pick if needed, just need to understand where > > is the 1.11.x branch if we have one (or how is the procedure for this > > in an ongoing release). > > https://github.com/apache/parquet-mr/pull/638 > > > > > > > > On Fri, May 31, 2019 at 6:25 PM Ryan Blue <[email protected]> > > wrote: > > > > > > I think we can add that one. > > > > > > On Fri, May 31, 2019 at 9:18 AM Michael Heuer <[email protected]> wrote: > > > > > > > Might > > > > > > > > https://github.com/apache/parquet-mr/pull/560 > > > > > > > > be included in the next 1.11.0 release candidate? > > > > > > > > michael > > > > > > > > > > > > On May 31, 2019, at 11:09 AM, Ryan Blue <[email protected]> > > wrote: > > > > > > > > I'm hoping to find some time to get a release candidate out next week > > or > > > > the week after. > > > > > > > > On Fri, May 31, 2019 at 4:32 AM Ismaël Mejía <[email protected]> > > wrote: > > > > > > > > Hello, > > > > > > > > What is the status of this, any chance we can have 1.11.0 out soon? > > > > > > > > Thanks, > > > > Ismaël > > > > > > > > > > > > On 2019/03/19 14:00:30, Gabor Szadovszky <[email protected]> wrote: > > > > > > > > Dear Parquet Users and Developers, > > > > > > > > I propose the following RC to be released as the official Apache > > > > Parquet 1.11.0 release: > > > > > > > > The commit id is 9756b0e2b35437a09716707a81e2ac0c187112ed > > > > * This corresponds to the tag: apache-parquet-1.11.0 > > > > * > > > > > > > > > > > > > > https://github.com/apache/parquet-mr/tree/9756b0e2b35437a09716707a81e2ac0c187112ed > > > > > > > > > > > > The release tarball, signature, and checksums are here: > > > > * > > > > > > > > > > https://dist.apache.org/repos/dist/dev/parquet/apache-parquet-1.11.0-rc6/ > > > > > > > > > > > > You can find the KEYS file here: > > > > * https://dist.apache.org/repos/dist/dev/parquet/KEYS > > > > > > > > Binary artifacts are staged in Nexus here: > > > > * > > > > > > > > > > > > > > https://repository.apache.org/content/groups/staging/org/apache/parquet/parquet/1.11.0/ > > > > > > > > > > > > This release includes the following new features: > > > > - PARQUET-1201 - Column indexes > > > > - PARQUET-1253 - Support for new logical type representation > > > > - PARQUET-1381 - Add merge blocks command to parquet-tools > > > > - PARQUET-1388 - Nanosecond precision time and timestamp - parquet-mr > > > > > > > > The release also includes bug fixes, including: > > > > - PARQUET-1472: Dictionary filter fails on FIXED_LEN_BYTE_ARRAY. > > > > - PARQUET-1510: Fix notEq for optional columns with null values. > > > > - PARQUET-1533: TestSnappy() throws OOM exception with Parquet-1485 > > > > > > > > change > > > > > > > > - PARQUET-1531: Page row count limit causes empty pages to be written > > > > > > > > from > > > > > > > > MessageColumnIO > > > > - PARQUET-1544: Possible over-shading of modules > > > > > > > > The following change has been reverted so it is not part of any public > > > > release: > > > > - PARQUET-1381: Add merge blocks command to parquet-tools > > > > > > > > Please download, verify, and test. The vote will be open for at least > > 72 > > > > hours. > > > > > > > > Thanks, > > > > Gabor > > > > > > > > > > > > > > > > > > > > -- > > > > Ryan Blue > > > > Software Engineer > > > > Netflix > > > > > > > > > > > > > > > > > > -- > > > Ryan Blue > > > Software Engineer > > > Netflix > >
