Re: [VOTE] Release Apache Parquet 1.8.2 RC1

[Michael Heuer]

Per comment
https://github.com/bigdatagenomics/adam/pull/1360#issuecomment-274681650

and Jenkins failure
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1757/HADOOP_VERSION=2.6.0,SCALAVER=2.10,SPARK_VERSION=1.5.2,label=centos/

when bumping our build to 1.8.2-rc1 unit tests succeed but we encounter
NoSuchMethodError errors at runtime in spark-submit.  Our build is a mess
of transitive dependencies and exclusions, so I haven't yet been able to
identify why bumping the parquet version would cause this problem.

I wanted to mention this here in case anyone else runs into it.

   michael


-- Forwarded message --
> From: Julien Le Dem <jul...@dremio.com>
> To: d...@parquet.apache.org, dev@spark.apache.org
> Cc:
> Date: Mon, 23 Jan 2017 11:43:40 -0800
> Subject: Re: [VOTE] Release Apache Parquet 1.8.2 RC1
> Hi Spark dev,
> Here is the voting thread for parquet 1.8.2 release.
> Cheng or someone else we would appreciate you verify it as well and reply
> to the thread.
>
> On Mon, Jan 23, 2017 at 11:40 AM, Julien Le Dem <jul...@dremio.com> wrote:
>
>> +1
>> Followed: https://cwiki.apache.org/confluence/display/PARQUET/How+To+
>> Verify+A+Release
>> checked sums, ran the build and tests.
>> We would appreciate someone from the Spark project (Cheng?) to verify the
>> release as well.
>> CC'ing spark
>>
>>
>> On Mon, Jan 23, 2017 at 10:15 AM, Ryan Blue <rb...@netflix.com.invalid>
>> wrote:
>>
>>> +1
>>>
>>> On Mon, Jan 23, 2017 at 10:15 AM, Daniel Weeks
>>> <dwe...@netflix.com.invalid>
>>> wrote:
>>>
>>> > +1 checked sums, built, tested
>>> >
>>> > On Mon, Jan 23, 2017 at 9:58 AM, Ryan Blue <rb...@netflix.com.invalid>
>>> > wrote:
>>> >
>>> > > Gabor, that md5 matches what I get. Are you sure you used the right
>>> file?
>>> > > It isn’t the same format that md5sum produces, but if you check the
>>> > octets
>>> > > the hash matches..
>>> > >
>>> > > [blue@work Downloads]$ md5sum apache-parquet-1.8.2.tar.gz
>>> > > b3743995bee616118c28f324598684ba  apache-parquet-1.8.2.tar.gz
>>> > >
>>> > > rb
>>> > > ​
>>> > >
>>> > > On Thu, Jan 19, 2017 at 8:06 AM, Gabor Szadovszky <
>>> > > gabor.szadovs...@cloudera.com> wrote:
>>> > >
>>> > > > Hi Ryan,
>>> > > >
>>> > > > I’ve downloaded the tar and checked the signature and the
>>> checksums.
>>> > SHA
>>> > > > and ASC are fine. MD5 is not and the content does not seem to be a
>>> > common
>>> > > > MD5 either:
>>> > > > apache-parquet-1.8.2.tar.gz: B3 74 39 95 BE E6 16 11  8C 28 F3 24
>>> 59 86
>>> > > 84
>>> > > > BA
>>> > > >
>>> > > > The artifacts on Nexus are good with all the related signatures and
>>> > > > checksums. The source zip properly contains the files from the repo
>>> > with
>>> > > > the tag apache-parquet-1.8.2.
>>> > > >
>>> > > > Regards,
>>> > > > Gabor
>>> > > >
>>> > > > > On 19 Jan 2017, at 04:09, Ryan Blue <b...@apache.org> wrote:
>>> > > > >
>>> > > > > Hi everyone,
>>> > > > >
>>> > > > > I propose the following RC to be released as official Apache
>>> Parquet
>>> > > > 1.8.2
>>> > > > > release.
>>> > > > >
>>> > > > > The commit id is c6522788629e590a53eb79874b95f6c3ff11f16c
>>> > > > > * This corresponds to the tag: apache-parquet-1.8.2
>>> > > > > * https://github.com/apache/parquet-mr/tree/c6522788
>>> > > > > *
>>> > > > > https://git-wip-us.apache.org/repos/asf/projects/repo?p=
>>> > > > parquet-mr.git=commit=c6522788
>>> > > > >
>>> > > > > The release tarball, signature, and checksums are here:
>>> > > > > * https://dist.apache.org/repos/dist/dev/parquet/apache-
>>> > > > parquet-1.8.2-rc1
>>> > > > >
>>> > > > > You can find the KEYS file here:
>>> > > > > * https://dist.apache.org/repos/dist/dev/parquet/KEYS
>>> > > > >
>>> > > > > Binary artifacts are staged in Nexus here:
>>> > > > > *
>>> > > > > https://repository.apache.org/content/groups/staging/org/
>>> > > > apache/parquet/parquet/1.8.2/
>>> > > > >
>>> > > > > This is a patch release with backports from the master branch.
>>> For a
>>> > > > > detailed summary, see the spreadsheet here:
>>> > > > >
>>> > > > > *
>>> > > > > https://docs.google.com/spreadsheets/d/1NAuY3c77Egs6REu-
>>> > > > UVkQqPswpVYVgZTTnY3bM0SPVRs/edit#gid=0
>>> > > > >
>>> > > > > Please download, verify, and test.
>>> > > > >
>>> > > > > Please vote by the end of Monday, 18 January.
>>> > > > >
>>> > > > > [ ] +1 Release this as Apache Parquet 1.8.2
>>> > > > > [ ] +0
>>> > > > > [ ] -1 Do not release this because...
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > --
>>> > > > > Ryan Blue
>>> > > >
>>> > > >
>>> > >
>>> > >
>>> > > --
>>> > > Ryan Blue
>>> > > Software Engineer
>>> > > Netflix
>>> > >
>>> >
>>>
>>>
>>>
>>> --
>>> Ryan Blue
>>> Software Engineer
>>> Netflix
>>>
>>
>>
>>
>> --
>> Julien
>>
>
>
>
> --
> Julien
>
>

Re: [VOTE] Release Apache Parquet 1.8.2 RC1

[Julien Le Dem]

Thank you Cheng!

On Mon, Jan 23, 2017 at 12:02 PM, Cheng Lian  wrote:

> Sorry for being late, I'm building a Spark branch based on the most recent
> master to test out 1.8.2-rc1, will post my result here ASAP.
>
> Cheng
>
> On 1/23/17 11:43 AM, Julien Le Dem wrote:
>
> Hi Spark dev,
> Here is the voting thread for parquet 1.8.2 release.
> Cheng or someone else we would appreciate you verify it as well and reply
> to the thread.
>
> On Mon, Jan 23, 2017 at 11:40 AM, Julien Le Dem  wrote:
>
>> +1
>> Followed: https://cwiki.apache.org/confluence/display/PARQUET/How+To+
>> Verify+A+Release
>> checked sums, ran the build and tests.
>> We would appreciate someone from the Spark project (Cheng?) to verify the
>> release as well.
>> CC'ing spark
>>
>>
>> On Mon, Jan 23, 2017 at 10:15 AM, Ryan Blue 
>> wrote:
>>
>>> +1
>>>
>>> On Mon, Jan 23, 2017 at 10:15 AM, Daniel Weeks
>>>  
>>> wrote:
>>>
>>> > +1 checked sums, built, tested
>>> >
>>> > On Mon, Jan 23, 2017 at 9:58 AM, Ryan Blue 
>>> 
>>> > wrote:
>>> >
>>> > > Gabor, that md5 matches what I get. Are you sure you used the right
>>> file?
>>> > > It isn’t the same format that md5sum produces, but if you check the
>>> > octets
>>> > > the hash matches..
>>> > >
>>> > > [blue@work Downloads]$ md5sum apache-parquet-1.8.2.tar.gz
>>> > > b3743995bee616118c28f324598684ba  apache-parquet-1.8.2.tar.gz
>>> > >
>>> > > rb
>>> > > ​
>>> > >
>>> > > On Thu, Jan 19, 2017 at 8:06 AM, Gabor Szadovszky <
>>> > > gabor.szadovs...@cloudera.com> wrote:
>>> > >
>>> > > > Hi Ryan,
>>> > > >
>>> > > > I’ve downloaded the tar and checked the signature and the
>>> checksums.
>>> > SHA
>>> > > > and ASC are fine. MD5 is not and the content does not seem to be a
>>> > common
>>> > > > MD5 either:
>>> > > > apache-parquet-1.8.2.tar.gz: B3 74 39 95 BE E6 16 11  8C 28 F3 24
>>> 59 86
>>> > > 84
>>> > > > BA
>>> > > >
>>> > > > The artifacts on Nexus are good with all the related signatures and
>>> > > > checksums. The source zip properly contains the files from the repo
>>> > with
>>> > > > the tag apache-parquet-1.8.2.
>>> > > >
>>> > > > Regards,
>>> > > > Gabor
>>> > > >
>>> > > > > On 19 Jan 2017, at 04:09, Ryan Blue  wrote:
>>> > > > >
>>> > > > > Hi everyone,
>>> > > > >
>>> > > > > I propose the following RC to be released as official Apache
>>> Parquet
>>> > > > 1.8.2
>>> > > > > release.
>>> > > > >
>>> > > > > The commit id is c6522788629e590a53eb79874b95f6c3ff11f16c
>>> > > > > * This corresponds to the tag: apache-parquet-1.8.2
>>> > > > > * https://github.com/apache/parquet-mr/tree/c6522788
>>> > > > > *
>>> > > > > https://git-wip-us.apache.org/repos/asf/projects/repo?p=
>>> > > > parquet-mr.git=commit=c6522788
>>> > > > >
>>> > > > > The release tarball, signature, and checksums are here:
>>> > > > > * https://dist.apache.org/repos/dist/dev/parquet/apache-
>>> > > > parquet-1.8.2-rc1
>>> > > > >
>>> > > > > You can find the KEYS file here:
>>> > > > > * https://dist.apache.org/repos/dist/dev/parquet/KEYS
>>> > > > >
>>> > > > > Binary artifacts are staged in Nexus here:
>>> > > > > *
>>> > > > > https://repository.apache.org/content/groups/staging/org/
>>> > > > apache/parquet/parquet/1.8.2/
>>> > > > >
>>> > > > > This is a patch release with backports from the master branch.
>>> For a
>>> > > > > detailed summary, see the spreadsheet here:
>>> > > > >
>>> > > > > *
>>> > > > > https://docs.google.com/spreadsheets/d/1NAuY3c77Egs6REu-
>>> > > > UVkQqPswpVYVgZTTnY3bM0SPVRs/edit#gid=0
>>> > > > >
>>> > > > > Please download, verify, and test.
>>> > > > >
>>> > > > > Please vote by the end of Monday, 18 January.
>>> > > > >
>>> > > > > [ ] +1 Release this as Apache Parquet 1.8.2
>>> > > > > [ ] +0
>>> > > > > [ ] -1 Do not release this because...
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > --
>>> > > > > Ryan Blue
>>> > > >
>>> > > >
>>> > >
>>> > >
>>> > > --
>>> > > Ryan Blue
>>> > > Software Engineer
>>> > > Netflix
>>> > >
>>> >
>>>
>>>
>>>
>>> --
>>> Ryan Blue
>>> Software Engineer
>>> Netflix
>>>
>>
>>
>>
>> --
>> Julien
>>
>
>
>
> --
> Julien
>
>
>


-- 
Julien

Re: [VOTE] Release Apache Parquet 1.8.2 RC1

[Cheng Lian]

Sorry for being late, I'm building a Spark branch based on the most 
recent master to test out 1.8.2-rc1, will post my result here ASAP.


Cheng


On 1/23/17 11:43 AM, Julien Le Dem wrote:

Hi Spark dev,
Here is the voting thread for parquet 1.8.2 release.
Cheng or someone else we would appreciate you verify it as well and 
reply to the thread.


On Mon, Jan 23, 2017 at 11:40 AM, Julien Le Dem > wrote:


+1
Followed:
https://cwiki.apache.org/confluence/display/PARQUET/How+To+Verify+A+Release


checked sums, ran the build and tests.
We would appreciate someone from the Spark project (Cheng?) to
verify the release as well.
CC'ing spark


On Mon, Jan 23, 2017 at 10:15 AM, Ryan Blue
> wrote:

+1

On Mon, Jan 23, 2017 at 10:15 AM, Daniel Weeks

wrote:

> +1 checked sums, built, tested
>
> On Mon, Jan 23, 2017 at 9:58 AM, Ryan Blue

> wrote:
>
> > Gabor, that md5 matches what I get. Are you sure you used
the right file?
> > It isn’t the same format that md5sum produces, but if you
check the
> octets
> > the hash matches..
> >
> > [blue@work Downloads]$ md5sum apache-parquet-1.8.2.tar.gz
> > b3743995bee616118c28f324598684ba apache-parquet-1.8.2.tar.gz
> >
> > rb
> > ​
> >
> > On Thu, Jan 19, 2017 at 8:06 AM, Gabor Szadovszky <
> > gabor.szadovs...@cloudera.com
> wrote:
> >
> > > Hi Ryan,
> > >
> > > I’ve downloaded the tar and checked the signature and
the checksums.
> SHA
> > > and ASC are fine. MD5 is not and the content does not
seem to be a
> common
> > > MD5 either:
> > > apache-parquet-1.8.2.tar.gz: B3 74 39 95 BE E6 16 11  8C
28 F3 24 59 86
> > 84
> > > BA
> > >
> > > The artifacts on Nexus are good with all the related
signatures and
> > > checksums. The source zip properly contains the files
from the repo
> with
> > > the tag apache-parquet-1.8.2.
> > >
> > > Regards,
> > > Gabor
> > >
> > > > On 19 Jan 2017, at 04:09, Ryan Blue > wrote:
> > > >
> > > > Hi everyone,
> > > >
> > > > I propose the following RC to be released as official
Apache Parquet
> > > 1.8.2
> > > > release.
> > > >
> > > > The commit id is c6522788629e590a53eb79874b95f6c3ff11f16c
> > > > * This corresponds to the tag: apache-parquet-1.8.2
> > > > * https://github.com/apache/parquet-mr/tree/c6522788

> > > > *
> > > >
https://git-wip-us.apache.org/repos/asf/projects/repo?p=

> > > parquet-mr.git=commit=c6522788
> > > >
> > > > The release tarball, signature, and checksums are here:
> > > > *
https://dist.apache.org/repos/dist/dev/parquet/apache-

> > > parquet-1.8.2-rc1
> > > >
> > > > You can find the KEYS file here:
> > > > * https://dist.apache.org/repos/dist/dev/parquet/KEYS

> > > >
> > > > Binary artifacts are staged in Nexus here:
> > > > *
> > > >
https://repository.apache.org/content/groups/staging/org/

> > > apache/parquet/parquet/1.8.2/
> > > >
> > > > This is a patch release with backports from the master
branch. For a
> > > > detailed summary, see the spreadsheet here:
> > > >
> > > > *
> > > >
https://docs.google.com/spreadsheets/d/1NAuY3c77Egs6REu-

> > > UVkQqPswpVYVgZTTnY3bM0SPVRs/edit#gid=0
> > > >
> > > > Please download, verify, and test.
> > > >
> > > > Please vote by the end of Monday, 18 January.
> > > >
> > > > [ ] +1 Release this as Apache Parquet 1.8.2
> > > > [ ] +0
> > > > [ ] -1 Do not release this because...
> > > >
> > > >
> > > >
> > > > --
> > > > Ryan Blue
> > >
> > >
> >
> >
> > --
> > Ryan Blue
> > 

Re: [VOTE] Release Apache Parquet 1.8.2 RC1

[Julien Le Dem]

Hi Spark dev,
Here is the voting thread for parquet 1.8.2 release.
Cheng or someone else we would appreciate you verify it as well and reply
to the thread.

On Mon, Jan 23, 2017 at 11:40 AM, Julien Le Dem  wrote:

> +1
> Followed: https://cwiki.apache.org/confluence/display/PARQUET/
> How+To+Verify+A+Release
> checked sums, ran the build and tests.
> We would appreciate someone from the Spark project (Cheng?) to verify the
> release as well.
> CC'ing spark
>
>
> On Mon, Jan 23, 2017 at 10:15 AM, Ryan Blue 
> wrote:
>
>> +1
>>
>> On Mon, Jan 23, 2017 at 10:15 AM, Daniel Weeks > >
>> wrote:
>>
>> > +1 checked sums, built, tested
>> >
>> > On Mon, Jan 23, 2017 at 9:58 AM, Ryan Blue 
>> > wrote:
>> >
>> > > Gabor, that md5 matches what I get. Are you sure you used the right
>> file?
>> > > It isn’t the same format that md5sum produces, but if you check the
>> > octets
>> > > the hash matches..
>> > >
>> > > [blue@work Downloads]$ md5sum apache-parquet-1.8.2.tar.gz
>> > > b3743995bee616118c28f324598684ba  apache-parquet-1.8.2.tar.gz
>> > >
>> > > rb
>> > > ​
>> > >
>> > > On Thu, Jan 19, 2017 at 8:06 AM, Gabor Szadovszky <
>> > > gabor.szadovs...@cloudera.com> wrote:
>> > >
>> > > > Hi Ryan,
>> > > >
>> > > > I’ve downloaded the tar and checked the signature and the checksums.
>> > SHA
>> > > > and ASC are fine. MD5 is not and the content does not seem to be a
>> > common
>> > > > MD5 either:
>> > > > apache-parquet-1.8.2.tar.gz: B3 74 39 95 BE E6 16 11  8C 28 F3 24
>> 59 86
>> > > 84
>> > > > BA
>> > > >
>> > > > The artifacts on Nexus are good with all the related signatures and
>> > > > checksums. The source zip properly contains the files from the repo
>> > with
>> > > > the tag apache-parquet-1.8.2.
>> > > >
>> > > > Regards,
>> > > > Gabor
>> > > >
>> > > > > On 19 Jan 2017, at 04:09, Ryan Blue  wrote:
>> > > > >
>> > > > > Hi everyone,
>> > > > >
>> > > > > I propose the following RC to be released as official Apache
>> Parquet
>> > > > 1.8.2
>> > > > > release.
>> > > > >
>> > > > > The commit id is c6522788629e590a53eb79874b95f6c3ff11f16c
>> > > > > * This corresponds to the tag: apache-parquet-1.8.2
>> > > > > * https://github.com/apache/parquet-mr/tree/c6522788
>> > > > > *
>> > > > > https://git-wip-us.apache.org/repos/asf/projects/repo?p=
>> > > > parquet-mr.git=commit=c6522788
>> > > > >
>> > > > > The release tarball, signature, and checksums are here:
>> > > > > * https://dist.apache.org/repos/dist/dev/parquet/apache-
>> > > > parquet-1.8.2-rc1
>> > > > >
>> > > > > You can find the KEYS file here:
>> > > > > * https://dist.apache.org/repos/dist/dev/parquet/KEYS
>> > > > >
>> > > > > Binary artifacts are staged in Nexus here:
>> > > > > *
>> > > > > https://repository.apache.org/content/groups/staging/org/
>> > > > apache/parquet/parquet/1.8.2/
>> > > > >
>> > > > > This is a patch release with backports from the master branch.
>> For a
>> > > > > detailed summary, see the spreadsheet here:
>> > > > >
>> > > > > *
>> > > > > https://docs.google.com/spreadsheets/d/1NAuY3c77Egs6REu-
>> > > > UVkQqPswpVYVgZTTnY3bM0SPVRs/edit#gid=0
>> > > > >
>> > > > > Please download, verify, and test.
>> > > > >
>> > > > > Please vote by the end of Monday, 18 January.
>> > > > >
>> > > > > [ ] +1 Release this as Apache Parquet 1.8.2
>> > > > > [ ] +0
>> > > > > [ ] -1 Do not release this because...
>> > > > >
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Ryan Blue
>> > > >
>> > > >
>> > >
>> > >
>> > > --
>> > > Ryan Blue
>> > > Software Engineer
>> > > Netflix
>> > >
>> >
>>
>>
>>
>> --
>> Ryan Blue
>> Software Engineer
>> Netflix
>>
>
>
>
> --
> Julien
>



-- 
Julien