Re: [DISCUSS] rename parquet-mr to parquet-java?

2024-05-17 Thread Wang, Yuming 
+10086

From: Uwe L. Korn 
Date: Thursday, May 16, 2024 at 15:41
To: dev@parquet.apache.org 
Subject: Re: [DISCUSS] rename parquet-mr to parquet-java?
External Email

very heavy +1

This would help a lot.

On Thu, May 16, 2024, at 4:19 AM, Gang Wu wrote:
> +1 on renaming the repo to reduce confusion.
>
> However, the java library still uses the "parquet-mr" prefix to write its
> application version [1] and it is consumed by downstream projects like
> parquet-cpp [2] as well.
>
> [1]
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsearch%3Fq%3Drepo%253Aapache%252Fparquet-mr%2Bparquet-mr%2Blanguage%253AJava%26type%3Dcode%26l%3DJava&data=05%7C02%7Cyumwang%40ebay.com%7Cfed824d59ca84cb3004408dc757b882c%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638514420629473555%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=E4AZ8hEbQyCLy3aXAU2umohUlTCksqHVO5Imfc%2BM6p0%3D&reserved=0
> [2]
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsearch%3Fq%3Drepo%253Aapache%252Farrow%2Bparquet-mr%2Blanguage%253AC%252B%252B%2B%26type%3Dcode&data=05%7C02%7Cyumwang%40ebay.com%7Cfed824d59ca84cb3004408dc757b882c%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638514420629484103%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=cuaTrenLP4WKex0Mbdk7DbbhzcEP45jhqwj5swRZ5Pk%3D&reserved=0
>
> Best,
> Gang
>
> On Thu, May 16, 2024 at 12:47 AM Vinoo Ganesh 
> wrote:
>
>> +1, I think this will make things a lot clearer! (non-binding)
>>
>> 
>>
>>
>> On Wed, May 15, 2024 at 12:36 PM Jacques Nadeau 
>> wrote:
>>
>> > +1000
>> >
>> > On Wed, May 15, 2024 at 6:30 AM Andrew Lamb 
>> > wrote:
>> >
>> > > Julien had a great suggestion[1] to  rename the parquet-mr repository
>> to
>> > > parquet-java to reduce confusion about its content.
>> > >
>> > >  > This looks great. Thank you for taking the initiative. Hadoop is not
>> > > required indeed. Perhaps at some point we should rename parquet-mr to
>> > > parquet-java?
>> > >
>> > > Having just renamed 
>> > > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Farrow-datafusion&data=05%7C02%7Cyumwang%40ebay.com%7Cfed824d59ca84cb3004408dc757b882c%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638514420629491325%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=ANZoBX%2B7w4Uu%2BCvBfiuUBLRXqAIF5KDSmtm%2BCtQmroc%3D&reserved=0
>> > >  to
>> > > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fdatafusion&data=05%7C02%7Cyumwang%40ebay.com%7Cfed824d59ca84cb3004408dc757b882c%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638514420629496155%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=CLaLQrZNT761zudqaoMkY2EeC%2F5MGdOeY5veMRe5WcI%3D&reserved=0
>> > >  I think this would be a
>> relatively
>> > > painless experience as all existing links still work
>> > >
>> > > I filed a ticket here
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FPARQUET-2475&data=05%7C02%7Cyumwang%40ebay.com%7Cfed824d59ca84cb3004408dc757b882c%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638514420629500280%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=0E9HcvNlNqJoXCfCI7Nzl0Frphoqnlvu9gWb%2Fd5%2BOXM%3D&reserved=0
>> > >
>> > > Thoughts?
>> > > Andrew
>> > >
>> > > [1]
>> > >
>> >
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-site%2Fpull%2F59%23pullrequestreview-2056038304&data=05%7C02%7Cyumwang%40ebay.com%7Cfed824d59ca84cb3004408dc757b882c%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638514420629504472%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=fTAB%2FtTFPGyqrJR11uCGYfjGUcmMAJL8q7LykgK%2BzXM%3D&reserved=0
>> > >
>> >
>>

Re: [VOTE] Release Apache Parquet 1.13.0 RC0

2023-04-04 Thread Wang, Yuming 
Yes, otherwise downstream users would need to compile it manually: 
https://github.com/apache/parquet-mr/blob/master/README.md?plain=1#L98


From: Gang Wu 
Date: Tuesday, April 4, 2023 at 14:44
To: dev@parquet.apache.org 
Subject: Re: [VOTE] Release Apache Parquet 1.13.0 RC0
External Email

Hi Yuming,

This is by purpose as it requires Java 17 to enable it.

Best,
Gang

On Tue, Apr 4, 2023 at 2:35 PM Wang, Yuming 
wrote:

> Binary artifacts do not include parquet-encoding-vector. Could we include
> it?
>
> From: Xinli shang 
> Date: Tuesday, April 4, 2023 at 09:54
> To: dev@parquet.apache.org 
> Subject: Re: [VOTE] Release Apache Parquet 1.13.0 RC0
> External Email
>
> +1
>
> Verified checksum and signature, and ran internal tests.
>
> Gang, thanks a lot for leading this effort!
>
> On Mon, Apr 3, 2023 at 12:06 AM Gábor Szádovszky  wrote:
>
> > Verified checksum and signature, diffed tarball and repo content,
> > build/unit tests pass.
> > +1 (binding) for releasing this content as 1.13.0
> >
> > NOTE: It is completely fine or even a good practice to release the first
> > minor release from its separate branch (instead of master). Do not forget
> > to merge back CHANGES.md and the new version numbers update
> > (1.14.0-SNAPSHOT) to master, please.
> >
> > Thank you again, Gang for working on this release!
> >
> > On 2023/04/03 05:43:58 "Wang, Yuming" wrote:
> > > +1. Tested this release through Apache Spark UT:
> >
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fspark%2Fpull%2F40555&data=05%7C01%7Cyumwang%40ebay.com%7Cc13dee92c13f4b7a2ed008db34d80ac3%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638161874734919985%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=E4PDu1tqja6RUK93bqFmR2Uk%2FkKUsCfWgUirQFl8zms%3D&reserved=0
> > >
> > > From: Gang Wu 
> > > Date: Monday, April 3, 2023 at 00:40
> > > To: dev@parquet.apache.org 
> > > Subject: [VOTE] Release Apache Parquet 1.13.0 RC0
> > > External Email
> > >
> > > Hi everyone,
> > >
> > > I propose the following RC to be released as the official Apache
> Parquet
> > > 1.13.0 release.
> > >
> > > The commit id is 2e369ed173f66f057c296e63c1bc31d77f294f41
> > > * This corresponds to the tag: apache-parquet-1.13.0-rc0
> > > *
> > >
> >
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Ftree%2F2e369ed173f66f057c296e63c1bc31d77f294f41&data=05%7C01%7Cyumwang%40ebay.com%7Cc13dee92c13f4b7a2ed008db34d80ac3%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638161874734919985%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MmVZsn6S4V%2FYRyZiRChqdtfah1kds7azMvkzZzVwrp0%3D&reserved=0
> > >
> > > The release tarball, signature, and checksums are here:
> > > *
> >
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2Fapache-parquet-1.13.0-rc0&data=05%7C01%7Cyumwang%40ebay.com%7Cc13dee92c13f4b7a2ed008db34d80ac3%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638161874734919985%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=DipqzUshcedQJvI5CwEkBN0%2FTr21CNbMUSIFQ6xzpCU%3D&reserved=0
> > >
> > > You can find the KEYS file here:
> > > *
> >
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdownloads.apache.org%2Fparquet%2FKEYS&data=05%7C01%7Cyumwang%40ebay.com%7Cc13dee92c13f4b7a2ed008db34d80ac3%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638161874734919985%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=FjTVGOLU5l1Osp0bnRxnCNfLl8Ytr44fLYNWVG3HxKA%3D&reserved=0
> > >
> > > Binary artifacts are staged in Nexus here:
> > > *
> >
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Fgroups%2Fstaging%2Forg%2Fapache%2Fparquet%2F&data=05%7C01%7Cyumwang%40ebay.com%7Cc13dee92c13f4b7a2ed008db34d80ac3%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638161874734919985%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=pU0hPpV4QZ%2B%2FFMXVkAJ59BAWjJ%2F5osomW%2BVZGbdfOq0%3D&reserved=0
> > >
> > > This release includes important changes listed:
> > > *
> >
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Fblob%2Fpar

Re: [VOTE] Release Apache Parquet 1.13.0 RC0

2023-04-03 Thread Wang, Yuming 
Binary artifacts do not include parquet-encoding-vector. Could we include it?

From: Xinli shang 
Date: Tuesday, April 4, 2023 at 09:54
To: dev@parquet.apache.org 
Subject: Re: [VOTE] Release Apache Parquet 1.13.0 RC0
External Email

+1

Verified checksum and signature, and ran internal tests.

Gang, thanks a lot for leading this effort!

On Mon, Apr 3, 2023 at 12:06 AM Gábor Szádovszky  wrote:

> Verified checksum and signature, diffed tarball and repo content,
> build/unit tests pass.
> +1 (binding) for releasing this content as 1.13.0
>
> NOTE: It is completely fine or even a good practice to release the first
> minor release from its separate branch (instead of master). Do not forget
> to merge back CHANGES.md and the new version numbers update
> (1.14.0-SNAPSHOT) to master, please.
>
> Thank you again, Gang for working on this release!
>
> On 2023/04/03 05:43:58 "Wang, Yuming" wrote:
> > +1. Tested this release through Apache Spark UT:
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fspark%2Fpull%2F40555&data=05%7C01%7Cyumwang%40ebay.com%7C7eaf08f19d6648ef4c7008db34af9257%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638161700922905070%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=CW65m35%2BzJzfaD60208%2FZt%2BMw67sJerULfcz%2BQfWZO4%3D&reserved=0
> >
> > From: Gang Wu 
> > Date: Monday, April 3, 2023 at 00:40
> > To: dev@parquet.apache.org 
> > Subject: [VOTE] Release Apache Parquet 1.13.0 RC0
> > External Email
> >
> > Hi everyone,
> >
> > I propose the following RC to be released as the official Apache Parquet
> > 1.13.0 release.
> >
> > The commit id is 2e369ed173f66f057c296e63c1bc31d77f294f41
> > * This corresponds to the tag: apache-parquet-1.13.0-rc0
> > *
> >
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Ftree%2F2e369ed173f66f057c296e63c1bc31d77f294f41&data=05%7C01%7Cyumwang%40ebay.com%7C7eaf08f19d6648ef4c7008db34af9257%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638161700922905070%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=fM8vfwq6AMe2jxQqRBXY7DlsGd%2F98psl3AWn01YjB3Y%3D&reserved=0
> >
> > The release tarball, signature, and checksums are here:
> > *
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2Fapache-parquet-1.13.0-rc0&data=05%7C01%7Cyumwang%40ebay.com%7C7eaf08f19d6648ef4c7008db34af9257%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638161700922905070%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=JkGpHmY5msrUqhNLkpNyRMEtR9OSZNxLaRkKBMG7dHA%3D&reserved=0
> >
> > You can find the KEYS file here:
> > *
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdownloads.apache.org%2Fparquet%2FKEYS&data=05%7C01%7Cyumwang%40ebay.com%7C7eaf08f19d6648ef4c7008db34af9257%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638161700922905070%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=i5DW8BkTTVKiWIM174qsXgiiDR9e5SVnz8xrc2KDTIU%3D&reserved=0
> >
> > Binary artifacts are staged in Nexus here:
> > *
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Fgroups%2Fstaging%2Forg%2Fapache%2Fparquet%2F&data=05%7C01%7Cyumwang%40ebay.com%7C7eaf08f19d6648ef4c7008db34af9257%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638161700922905070%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=lKSo7ddjUlnTOiyebulbEXfAU5BUZJZ4u6FtZWmanF8%3D&reserved=0
> >
> > This release includes important changes listed:
> > *
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Fblob%2Fparquet-1.13.x%2FCHANGES.md&data=05%7C01%7Cyumwang%40ebay.com%7C7eaf08f19d6648ef4c7008db34af9257%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638161700922905070%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=5BhyoYV5h5Yl5Z4uhmhYWROwU4nzQuqkaXjWQtmzf5U%3D&reserved=0
> >
> > Please download, verify, and test.
> >
> > Please vote in the next 72 hours.
> >
> > [ ] +1 Release this as Apache Parquet 1.13.0
> > [ ] +0
> > [ ] -1 Do not release this because...
> >
> > Best regards,
> > Gang
> >
>


--
Xinli Shang

Re: [VOTE] Release Apache Parquet 1.13.0 RC0

2023-04-02 Thread Wang, Yuming 
+1. Tested this release through Apache Spark UT: 
https://github.com/apache/spark/pull/40555

From: Gang Wu 
Date: Monday, April 3, 2023 at 00:40
To: dev@parquet.apache.org 
Subject: [VOTE] Release Apache Parquet 1.13.0 RC0
External Email

Hi everyone,

I propose the following RC to be released as the official Apache Parquet
1.13.0 release.

The commit id is 2e369ed173f66f057c296e63c1bc31d77f294f41
* This corresponds to the tag: apache-parquet-1.13.0-rc0
*
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Ftree%2F2e369ed173f66f057c296e63c1bc31d77f294f41&data=05%7C01%7Cyumwang%40ebay.com%7C8ace28c601754ff8a0e908db3398f2b2%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638160504241192076%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=hwxVa%2FxkYd47gnxJg4PI5nSXPuuF%2FSIC1XqhwcDgbN0%3D&reserved=0

The release tarball, signature, and checksums are here:
* 
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2Fapache-parquet-1.13.0-rc0&data=05%7C01%7Cyumwang%40ebay.com%7C8ace28c601754ff8a0e908db3398f2b2%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638160504241192076%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=eb%2Fxey4DnprQyTxYRxdF201f7qz1zbm5berRDdVA3rY%3D&reserved=0

You can find the KEYS file here:
* 
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdownloads.apache.org%2Fparquet%2FKEYS&data=05%7C01%7Cyumwang%40ebay.com%7C8ace28c601754ff8a0e908db3398f2b2%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638160504241192076%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=pN8Xku%2BirF5nYcffdkJe4yh84mDFjjaVXewj0m8b1Kw%3D&reserved=0

Binary artifacts are staged in Nexus here:
* 
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Fgroups%2Fstaging%2Forg%2Fapache%2Fparquet%2F&data=05%7C01%7Cyumwang%40ebay.com%7C8ace28c601754ff8a0e908db3398f2b2%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638160504241192076%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qbyr5Y1EDslnqB8qi1CubNbPv9rATxpIoSbUmslaRIg%3D&reserved=0

This release includes important changes listed:
* 
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Fblob%2Fparquet-1.13.x%2FCHANGES.md&data=05%7C01%7Cyumwang%40ebay.com%7C8ace28c601754ff8a0e908db3398f2b2%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638160504241192076%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YmU60HCl776s6O4hvu%2FNFFXZY1ij9E0z9HquzmeJDxc%3D&reserved=0

Please download, verify, and test.

Please vote in the next 72 hours.

[ ] +1 Release this as Apache Parquet 1.13.0
[ ] +0
[ ] -1 Do not release this because...

Best regards,
Gang

Re: [VOTE] Release Apache Parquet 1.12.4 RC0

2023-03-26 Thread Wang, Yuming 
+1. Tested this release through Spark UT: 
https://github.com/apache/spark/pull/40555.


From: Gang Wu 
Date: Sunday, March 26, 2023 at 22:42
To: dev@parquet.apache.org 
Subject: [VOTE] Release Apache Parquet 1.12.4 RC0
External Email

Hi everyone,

I propose the following RC to be released as the official Apache Parquet
1.12.4 release.

The commit id is 22069e58494e7cb5d50e664c7ffa1cf1468404f8
* This corresponds to the tag: apache-parquet-1.12.4-rc0
*
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Ftree%2F22069e58494e7cb5d50e664c7ffa1cf1468404f8&data=05%7C01%7Cyumwang%40ebay.com%7Cc5216cd229664f939b6508db2e0855ed%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638154385567464296%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2Bny4R%2BgGQwIc3yMxsHfPh87YYTPhJ580UUoGV30WUQU%3D&reserved=0

The release tarball, signature, and checksums are here:
* 
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2Fapache-parquet-1.12.4-rc0&data=05%7C01%7Cyumwang%40ebay.com%7Cc5216cd229664f939b6508db2e0855ed%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638154385567464296%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qW7uIIvyamqkT7FbkBWvwKD1VnfeRWnKLUBpcVHXvck%3D&reserved=0

You can find the KEYS file here:
* 
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdownloads.apache.org%2Fparquet%2FKEYS&data=05%7C01%7Cyumwang%40ebay.com%7Cc5216cd229664f939b6508db2e0855ed%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638154385567464296%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=79Et30L9u4w4%2F%2B%2FTvPTpXEobOuvTV9XyVmapKC2qwoY%3D&reserved=0

Binary artifacts are staged in Nexus here:
* 
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Fgroups%2Fstaging%2Forg%2Fapache%2Fparquet%2F&data=05%7C01%7Cyumwang%40ebay.com%7Cc5216cd229664f939b6508db2e0855ed%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638154385567464296%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Z%2FhRa8zc5ZHhs15Epx7X%2BIUwQJI4MoyPMOgAIJemvHU%3D&reserved=0

This release includes important changes listed:
* 
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Fblob%2Fparquet-1.12.4%2FCHANGES.md&data=05%7C01%7Cyumwang%40ebay.com%7Cc5216cd229664f939b6508db2e0855ed%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638154385567464296%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=SXURCILyTz6SYb3iNPEnedkgjMk%2BA%2FLYHyS4TvT4bbM%3D&reserved=0

Please download, verify, and test.

Please vote in the next 72 hours.

[ ] +1 Release this as Apache Parquet 1.12.4
[ ] +0
[ ] -1 Do not release this because...

Best regards,
Gang

Re: [VOTE] Release Apache Parquet 1.12.3 RC1

2022-05-22 Thread Wang, Yuming 
+1. Tested through Spark: https://github.com/apache/spark/pull/36629

From: Gidon Gershinsky 
Date: Sunday, May 22, 2022 at 19:02
To: dev@parquet.apache.org 
Subject: Re: [VOTE] Release Apache Parquet 1.12.3 RC1
External Email

+1. Downloaded, verified and tested.

Cheers, Gidon


On Fri, May 20, 2022 at 8:49 PM Xinli shang  wrote:

> Hi everyone,
>
>
> I propose the following RC to be released as the official Apache Parquet
>  1.12.3 release.
>
>
> The commit id is f8dced182c4c1fbdec6ccb3185537b5a01e6ed6b
>
> * This corresponds to the tag: apache-parquet-1.12.3-rc1
>
> *
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Freleases%2Ftag%2Fapache-parquet-1.12.3-rc1&data=05%7C01%7Cyumwang%40ebay.com%7C23690f8480da452ddc8a08da3be296e3%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637888141765431335%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=S7wFdemHnELNQZWPHWSxfOcyz3pwBh1U67eGzLuSxXU%3D&reserved=0
>
>
> The release tarball, signature, and checksums are here:
>
> * 
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2Fapache-parquet-1.12.3-rc1&data=05%7C01%7Cyumwang%40ebay.com%7C23690f8480da452ddc8a08da3be296e3%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637888141765587985%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zz3qy8mFVR%2FuLrBykg7JKE6O9IOBXIys57n8SIykL4A%3D&reserved=0
>
>
> You can find the KEYS file here:
>
> * 
> *https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Frelease%2Fparquet%2FKEYS&data=05%7C01%7Cyumwang%40ebay.com%7C23690f8480da452ddc8a08da3be296e3%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637888141765587985%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=gIq%2Beqa2z%2BUANXBcz%2FRXFVAYwv%2BczYTS%2FB1uuuq84f4%3D&reserved=0
> *
>
>
> Binary artifacts are staged in Nexus here:
>
> * 
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Fgroups%2Fstaging%2Forg%2Fapache%2Fparquet%2F&data=05%7C01%7Cyumwang%40ebay.com%7C23690f8480da452ddc8a08da3be296e3%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637888141765587985%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Nel4PeDZ0dJjZfCFyTwSIloeeiGt30s33o75CL%2B8chc%3D&reserved=0
>
>
> This release includes important changes listed
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Fblob%2Fparquet-1.12.3%2FCHANGES.md&data=05%7C01%7Cyumwang%40ebay.com%7C23690f8480da452ddc8a08da3be296e3%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637888141765587985%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=kadO6pFwux8C18dL26GVoWbg4aOVTIxZtGUq6B5J0XM%3D&reserved=0.
>
>
> Please download, verify, and test.
>
>
> Please vote in the next 72 hours.
>
>
> [ ] +1 Release this as Apache Parquet 1.12.3
>
> [ ] +0
>
> [ ] -1 Do not release this because...
>
>
>
> 
>
> Xinli Shang
>
> PMC Chair of Apache Parquet
>
> TLM Uber Data Infra
>

Re: [VOTE] Release Apache Parquet 1.12.0 RC2

2021-02-27 Thread Wang, Yuming 
+1. Thank you Gabor. It works.

On 2021/2/26, 19:50, "Gabor Szadovszky"  wrote:

External Email

Thanks for testing. It seems that the previous (rc1) was stuck on the
staging maven repo. I've had to manually drop the previous upload. I don't
remember we've had to do such things before. Anyway, the referenced staging
maven repository now contains the rc2 build. Could you please try it again?

Thanks a lot,
Gabor

On Fri, Feb 26, 2021 at 1:30 AM Wang, Yuming 
wrote:

> -1. It still throws incompatible exceptions:
> Caused by: sbt.ForkMain$ForkError: java.lang.NoSuchMethodError:
> java.nio.ByteBuffer.rewind()Ljava/nio/ByteBuffer;
> at
> 
org.apache.parquet.hadoop.codec.SnappyCompressor.reset(SnappyCompressor.java:156)
> at
> 
org.apache.hadoop.io.compress.CodecPool.returnCompressor(CodecPool.java:210)
> at
> 
org.apache.parquet.hadoop.CodecFactory$HeapBytesCompressor.release(CodecFactory.java:177)
> at
> org.apache.parquet.hadoop.CodecFactory.release(CodecFactory.java:250)
> at
> 
org.apache.parquet.hadoop.ParquetRecordWriter.close(ParquetRecordWriter.java:168)
> at
> 
org.apache.spark.sql.execution.datasources.parquet.ParquetOutputWriter.close(ParquetOutputWriter.scala:41)
> at
> 
org.apache.spark.sql.execution.datasources.FileFormatDataWriter.releaseResources(FileFormatDataWriter.scala:58)
> at
> 
org.apache.spark.sql.execution.datasources.FileFormatDataWriter.commit(FileFormatDataWriter.scala:75)
> at
> 
org.apache.spark.sql.execution.datasources.FileFormatWriter$.$anonfun$executeTask$1(FileFormatWriter.scala:281)
> at
> 
org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1471)
> at
> 
org.apache.spark.sql.execution.datasources.FileFormatWriter$.executeTask(FileFormatWriter.scala:287)
> ... 9 more
>
> 
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fspark%2Fpull%2F31649&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Zm%2FlQaid9CeDDxig3RdWFzJ065lqOoT0s3T72hqFwc4%3D&reserved=0
> 
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fspark%2Fruns%2F1983121842%3Fcheck_suite_focus%3Dtrue&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2BV9GaP1SCYjkVyH42Rs1PbpFxLvS3AY9nVLJLXZL8tE%3D&reserved=0
>
>
> On 2021/2/25, 19:27, "Gabor Szadovszky"  wrote:
>
> External Email
>
> Hi everyone,
>
> I propose the following RC to be released as the official Apache
> Parquet
> 1.12.0 release.
>
> The commit id is d859b4435d139f1979f46cf63d2afdf32935ad51
> * This corresponds to the tag: apache-parquet-1.12.0-rc2
> *
>
> 
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Ftree%2Fd859b4435d139f1979f46cf63d2afdf32935ad51&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=S91VddHb1xV5XG5gMXSWU29UeF9H2yVbEjKv9koovYw%3D&reserved=0
>
> The release tarball, signature, and checksums are here:
> *
> 
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2Fapache-parquet-1.12.0-rc2&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Z6gftDJWkIdxqJH0LB6zbvEkf9Pb25uXOScmwQ%2BwwOM%3D&reserved=0
>
> You can find the KEYS file here:
> *
> 
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdownloads.apache.org%2Fparquet%2FKEYS&data=04%7C01%7Cyumwang%40ebay.com%7Cd34af2e6aa3749a295a408d8da4cc713%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637499370584039891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=7gMa%2Fw%2F0PUujCwTBV%2F34F0s4JZwwIy%2FWKaO8pLsXfkw%3D&reserved=0

Re: [VOTE] Release Apache Parquet 1.12.0 RC2

2021-02-25 Thread Wang, Yuming 
-1. It still throws incompatible exceptions:
Caused by: sbt.ForkMain$ForkError: java.lang.NoSuchMethodError: 
java.nio.ByteBuffer.rewind()Ljava/nio/ByteBuffer;
at 
org.apache.parquet.hadoop.codec.SnappyCompressor.reset(SnappyCompressor.java:156)
at 
org.apache.hadoop.io.compress.CodecPool.returnCompressor(CodecPool.java:210)
at 
org.apache.parquet.hadoop.CodecFactory$HeapBytesCompressor.release(CodecFactory.java:177)
at org.apache.parquet.hadoop.CodecFactory.release(CodecFactory.java:250)
at 
org.apache.parquet.hadoop.ParquetRecordWriter.close(ParquetRecordWriter.java:168)
at 
org.apache.spark.sql.execution.datasources.parquet.ParquetOutputWriter.close(ParquetOutputWriter.scala:41)
at 
org.apache.spark.sql.execution.datasources.FileFormatDataWriter.releaseResources(FileFormatDataWriter.scala:58)
at 
org.apache.spark.sql.execution.datasources.FileFormatDataWriter.commit(FileFormatDataWriter.scala:75)
at 
org.apache.spark.sql.execution.datasources.FileFormatWriter$.$anonfun$executeTask$1(FileFormatWriter.scala:281)
at 
org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1471)
at 
org.apache.spark.sql.execution.datasources.FileFormatWriter$.executeTask(FileFormatWriter.scala:287)
... 9 more

https://github.com/apache/spark/pull/31649
https://github.com/apache/spark/runs/1983121842?check_suite_focus=true


On 2021/2/25, 19:27, "Gabor Szadovszky"  wrote:

External Email

Hi everyone,

I propose the following RC to be released as the official Apache Parquet
1.12.0 release.

The commit id is d859b4435d139f1979f46cf63d2afdf32935ad51
* This corresponds to the tag: apache-parquet-1.12.0-rc2
*

https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Ftree%2Fd859b4435d139f1979f46cf63d2afdf32935ad51&data=04%7C01%7Cyumwang%40ebay.com%7Cf6486fad02f642f3987008d8d980476f%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637498492264763416%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=BRDSPrv9JNQVY3FEgNd%2B3v%2FUNj3nYnYwRFxH7P%2BeqfI%3D&reserved=0

The release tarball, signature, and checksums are here:
* 
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2Fapache-parquet-1.12.0-rc2&data=04%7C01%7Cyumwang%40ebay.com%7Cf6486fad02f642f3987008d8d980476f%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637498492264763416%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=6I9cfSlW0hzN5vFtMv6%2FnTb11v%2BMolcoENGod0pwuGk%3D&reserved=0

You can find the KEYS file here:
* 
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdownloads.apache.org%2Fparquet%2FKEYS&data=04%7C01%7Cyumwang%40ebay.com%7Cf6486fad02f642f3987008d8d980476f%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637498492264763416%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XBD6DvGcFxKVpQ0%2Bfq8aF6j0krO4rJjLqR76QYfjhNc%3D&reserved=0

Binary artifacts are staged in Nexus here:
* 
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Fgroups%2Fstaging%2Forg%2Fapache%2Fparquet%2F&data=04%7C01%7Cyumwang%40ebay.com%7Cf6486fad02f642f3987008d8d980476f%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637498492264763416%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=J05VEyDHfWax42zwNiIUaakX%2BEWBZvRdbfIm%2B4Ku51A%3D&reserved=0

For release content see:
*

https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Fblob%2Fapache-parquet-1.12.0-rc2%2FCHANGES.md&data=04%7C01%7Cyumwang%40ebay.com%7Cf6486fad02f642f3987008d8d980476f%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637498492264763416%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=yxOGzFI5DLpgekoNTGkDX4tYL6NLbLUrzZWS7X8LFTM%3D&reserved=0

Please download, verify, and test.

Please vote in the next 72 hours.

[ ] +1 Release this as Apache Parquet 1.12.0
[ ] +0
[ ] -1 Do not release this because...

Re: [VOTE] Release Apache Parquet 1.12.0 RC1

2021-01-29 Thread Wang, Yuming 

It seems there is something wrong with JDK 8:

java.lang.NoSuchMethodError: 
java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer;
at 
org.apache.parquet.bytes.CapacityByteArrayOutputStream.write(CapacityByteArrayOutputStream.java:197)
at 
org.apache.parquet.column.values.rle.RunLengthBitPackingHybridEncoder.writeOrAppendBitPackedRun(RunLengthBitPackingHybridEncoder.java:193)
at 
org.apache.parquet.column.values.rle.RunLengthBitPackingHybridEncoder.writeInt(RunLengthBitPackingHybridEncoder.java:179)
at 
org.apache.parquet.column.values.dictionary.DictionaryValuesWriter.getBytes(DictionaryValuesWriter.java:167)
at 
org.apache.parquet.column.values.fallback.FallbackValuesWriter.getBytes(FallbackValuesWriter.java:74)
at 
org.apache.parquet.column.impl.ColumnWriterV1.writePage(ColumnWriterV1.java:60)
at 
org.apache.parquet.column.impl.ColumnWriterBase.writePage(ColumnWriterBase.java:387)
at 
org.apache.parquet.column.impl.ColumnWriteStoreBase.sizeCheck(ColumnWriteStoreBase.java:235)
at 
org.apache.parquet.column.impl.ColumnWriteStoreBase.endRecord(ColumnWriteStoreBase.java:222)
at 
org.apache.parquet.column.impl.ColumnWriteStoreV1.endRecord(ColumnWriteStoreV1.java:29)
at 
org.apache.parquet.io.MessageColumnIO$MessageColumnIORecordConsumer.endMessage(MessageColumnIO.java:307)
at 
org.apache.spark.sql.execution.datasources.parquet.ParquetWriteSupport.consumeMessage(ParquetWriteSupport.scala:465)
at 
org.apache.spark.sql.execution.datasources.parquet.ParquetWriteSupport.write(ParquetWriteSupport.scala:148)
at 
org.apache.spark.sql.execution.datasources.parquet.ParquetWriteSupport.write(ParquetWriteSupport.scala:54)
at 
org.apache.parquet.hadoop.InternalParquetRecordWriter.write(InternalParquetRecordWriter.java:138)


On 2021/1/29, 18:01, "Gidon Gershinsky"  wrote:

External Email

Regarding the technical reason behind this addition - we needed it to
enable encryption in one of the writing paths.

Cheers, Gidon


On Thu, Jan 28, 2021 at 7:09 PM Aaron Niskode-Dossett
 wrote:

> My (non-binding) is that this is ok.  In a different Apache project, we
> didn't allow a change like that in minor versions and it delayed some key
> work by several months.
>
> On Thu, Jan 28, 2021 at 3:00 AM Gabor Szadovszky  wrote:
>
> > Thanks a lot, Fokko.
> >
> > Regarding the breaking change. We have the maven plugin japicmp executed
> in
> > the verify phase so I was curious why it did not catch this issue. It
> seems
> > the plugin allows source incompatible changes for minor version upgrades
> by
> > default. It sounds reasonable to me but I am curious about the opinion 
of
> > the community. See details about the plugin at
> > 
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsiom79.github.io%2Fjapicmp%2FMavenPlugin.html&data=04%7C01%7Cyumwang%40ebay.com%7Ca25ea17897d64584603408d8c43cd748%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637475112885259165%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=7FyRjo3idZOwtT5dRYRSwbC5oxcf3unhYuO72nroI74%3D&reserved=0.
 Search
> > for METHOD_ADDED_TO_INTERFACE to find info about the current issue.
> >
> > Cheers,
> > Gabor
> >
> >
> > On Wed, Jan 27, 2021 at 11:08 PM Driesprong, Fokko  >
> > wrote:
> >
> > > Thanks for running the release Gabor!
> > >
> > > The signature checks out:
> > > MacBook-Pro-van-Fokko:Downloads fokkodriesprong$ curl
> > > 
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdownloads.apache.org%2Fparquet%2FKEYS&data=04%7C01%7Cyumwang%40ebay.com%7Ca25ea17897d64584603408d8c43cd748%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637475112885259165%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=SxoDKeIRmWww%2FzRI9tYycdDFF9rQONh0iD4XwwVCYPE%3D&reserved=0
 > KEYS
> > >   % Total% Received % Xferd  Average Speed   TimeTime Time
> > >  Current
> > >  Dload  Upload   Total   SpentLeft
> > >  Speed
> > > 100 33082  100 330820 0   212k  0 --:--:-- --:--:--
> --:--:--
> > >  212k
> > > MacBook-Pro-van-Fokko:Downloads fokkodriesprong$ gpg --import  KEYS
> > > gpg: key 97D7E8647AE7E47B: 2 signatures not checked due to missing 
keys
> > > gpg: key 97D7E8647AE7E47B: public key "Julien Le Dem  >"
> > > imported
> > > gpg: key 7CD8278971F0F13B: 1 signature not checked due to a missing 
key
> > > gpg: key 7CD8278971F0F13B: public key "Tianshuo Deng <
> td...@twitter.com
> > >"
> > > imported
> > > gpg: key 4FB955854318F669: 8 signatures not checked due to missing 
keys
> > > gpg: key 4FB955854318F669: public key "Tom White (CODE SIGNING KEY) <
> > > t

Re: [VOTE] Release Apache Parquet 1.11.0 RC7

2019-11-13 Thread Wang, Yuming 
+1 
Tested Parquet 1.11.0 with Spark SQL module: build/sbt "sql/test-only" 
-Phadoop-3.2

On 2019/11/13, 21:33, "Gabor Szadovszky"  wrote:

Hi everyone,

I propose the following RC to be released as official Apache Parquet 1.11.0
release.

The commit id is 18519eb8e059865652eee3ff0e8593f126701da4
* This corresponds to the tag: apache-parquet-1.11.0-rc7
*

https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Ftree%2F18519eb8e059865652eee3ff0e8593f126701da4&data=02%7C01%7Cyumwang%40ebay.com%7C8d588ca5855842a94bed08d7683e1221%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637092488114756267&sdata=ToLFrTB9lU%2FGzH6UpXwy7PAY7kaupbyKAgdghESCfgg%3D&reserved=0

The release tarball, signature, and checksums are here:
* 
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2Fapache-parquet-1.11.0-rc7&data=02%7C01%7Cyumwang%40ebay.com%7C8d588ca5855842a94bed08d7683e1221%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637092488114756267&sdata=MPaHiYJT7ZcqreAYUkvDvZugthUhRPrySdXpN2ytT5k%3D&reserved=0

You can find the KEYS file here:
* 
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapache.org%2Fdist%2Fparquet%2FKEYS&data=02%7C01%7Cyumwang%40ebay.com%7C8d588ca5855842a94bed08d7683e1221%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637092488114756267&sdata=IwG4MUGsP2lVzlD4bwZUEPuEAPUg%2FHXRYtxc5CQupBM%3D&reserved=0

Binary artifacts are staged in Nexus here:
* 
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Fgroups%2Fstaging%2Forg%2Fapache%2Fparquet%2F&data=02%7C01%7Cyumwang%40ebay.com%7C8d588ca5855842a94bed08d7683e1221%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637092488114756267&sdata=lHtqLRQqQFwsyoaLSVaJuau5gxPKsCQFFVJaY8H0tZQ%3D&reserved=0

This release includes the changes listed at:

https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Fblob%2Fapache-parquet-1.11.0-rc7%2FCHANGES.md&data=02%7C01%7Cyumwang%40ebay.com%7C8d588ca5855842a94bed08d7683e1221%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637092488114756267&sdata=82BplI3bLAL6qArLHvVoYReZOk%2BboSP655rI8VX5Q5I%3D&reserved=0

Please download, verify, and test.

Please vote in the next 72 hours.

[ ] +1 Release this as Apache Parquet 1.11.0
[ ] +0
[ ] -1 Do not release this because...