Re: parquet-mr next release with PARQUET-1217?
Yeah, Spark would (I think) prefer a 1.8.3. I'd be happy to drive the release, but not clear if it's restricted to PMC members or committers to do (particularly, tagging in the repository probably requires write access). On 12 April 2018 at 14:48, Ryan Blue wrote: > I think it would be better to do a 1.8.2 (or is it 1.8.3?) because Spark is > on the 1.8 line. I doubt Spark is going to move to 1.9 now that 1.10 is > out. > > On Thu, Apr 12, 2018 at 2:43 PM, Julien Le Dem > wrote: > > > If someone wants a 1.9.1 it can be done, we'll need someone to own the > > release process though. > > > > On Tue, Apr 10, 2018 at 3:53 PM, Henry Robinson > wrote: > > > >> Thanks! Sorry to miss the vote - was AFK for a few days. I look forward > to > >> testing it out anyhow. > >> > >> On 5 April 2018 at 14:28, Ryan Blue wrote: > >> > >> > I just sent a vote for this. Took longer than expected because I had > to > >> > fix all of the javadoc warnings for java 8. Please test it out and > vote. > >> > > >> > On Fri, Mar 30, 2018 at 10:44 AM, Ryan Blue > wrote: > >> > > >> >> I have no plan for 1.9.1. > >> >> > >> >> On Fri, Mar 30, 2018 at 10:42 AM, Henry Robinson > > >> >> wrote: > >> >> > >> >>> Great! Do you know of any plans to do a 1.9.1? > >> >>> > >> >>> On 30 March 2018 at 09:35, Ryan Blue > >> wrote: > >> >>> > >> I'm planning on getting a 1.10.0 rc out today, if I don't find > >> problems > >> with the stats changes. > >> > >> On Thu, Mar 29, 2018 at 4:18 PM, Henry Robinson > >> wrote: > >> > >> > Hi all - > >> > > >> > While using Spark, I got hit by PARQUET-1217 today on some data > >> written by > >> > Impala. This is a pretty nasty bug, and one that affects Apache > >> Spark > >> right > >> > now because, AFAICT, there's no release to move to that contains > >> the > >> fix, > >> > and parquet-mr 1.9.0 is affected. There is a workaround, but it's > >> expensive > >> > in terms of lost performance. > >> > > >> > I'm new to the community, so wanted to see if there was a plan to > >> make a > >> > release (1.9.1?) in the near future. I'd rather that than have to > >> build > >> > short-term workarounds into Spark. > >> > > >> > Best, > >> > Henry > >> > > >> > >> > >> > >> -- > >> Ryan Blue > >> Software Engineer > >> Netflix > >> > >> >>> > >> >>> > >> >>> > >> >>> -- > >> >>> Henry Robinson > >> >>> Software Engineer > >> >>> Cloudera > >> >>> 415-994-6679 <(415)%20994-6679> > >> >>> > >> >> > >> >> > >> >> > >> >> -- > >> >> Ryan Blue > >> >> Software Engineer > >> >> Netflix > >> >> > >> > > >> > > >> > > >> > -- > >> > Ryan Blue > >> > Software Engineer > >> > Netflix > >> > > >> > > > > > > > -- > Ryan Blue > Software Engineer > Netflix >
Re: parquet-mr next release with PARQUET-1217?
I think it would be better to do a 1.8.2 (or is it 1.8.3?) because Spark is on the 1.8 line. I doubt Spark is going to move to 1.9 now that 1.10 is out. On Thu, Apr 12, 2018 at 2:43 PM, Julien Le Dem wrote: > If someone wants a 1.9.1 it can be done, we'll need someone to own the > release process though. > > On Tue, Apr 10, 2018 at 3:53 PM, Henry Robinson wrote: > >> Thanks! Sorry to miss the vote - was AFK for a few days. I look forward to >> testing it out anyhow. >> >> On 5 April 2018 at 14:28, Ryan Blue wrote: >> >> > I just sent a vote for this. Took longer than expected because I had to >> > fix all of the javadoc warnings for java 8. Please test it out and vote. >> > >> > On Fri, Mar 30, 2018 at 10:44 AM, Ryan Blue wrote: >> > >> >> I have no plan for 1.9.1. >> >> >> >> On Fri, Mar 30, 2018 at 10:42 AM, Henry Robinson >> >> wrote: >> >> >> >>> Great! Do you know of any plans to do a 1.9.1? >> >>> >> >>> On 30 March 2018 at 09:35, Ryan Blue >> wrote: >> >>> >> I'm planning on getting a 1.10.0 rc out today, if I don't find >> problems >> with the stats changes. >> >> On Thu, Mar 29, 2018 at 4:18 PM, Henry Robinson >> wrote: >> >> > Hi all - >> > >> > While using Spark, I got hit by PARQUET-1217 today on some data >> written by >> > Impala. This is a pretty nasty bug, and one that affects Apache >> Spark >> right >> > now because, AFAICT, there's no release to move to that contains >> the >> fix, >> > and parquet-mr 1.9.0 is affected. There is a workaround, but it's >> expensive >> > in terms of lost performance. >> > >> > I'm new to the community, so wanted to see if there was a plan to >> make a >> > release (1.9.1?) in the near future. I'd rather that than have to >> build >> > short-term workarounds into Spark. >> > >> > Best, >> > Henry >> > >> >> >> >> -- >> Ryan Blue >> Software Engineer >> Netflix >> >> >>> >> >>> >> >>> >> >>> -- >> >>> Henry Robinson >> >>> Software Engineer >> >>> Cloudera >> >>> 415-994-6679 <(415)%20994-6679> >> >>> >> >> >> >> >> >> >> >> -- >> >> Ryan Blue >> >> Software Engineer >> >> Netflix >> >> >> > >> > >> > >> > -- >> > Ryan Blue >> > Software Engineer >> > Netflix >> > >> > > -- Ryan Blue Software Engineer Netflix
Re: parquet-mr next release with PARQUET-1217?
If someone wants a 1.9.1 it can be done, we'll need someone to own the release process though. On Tue, Apr 10, 2018 at 3:53 PM, Henry Robinson wrote: > Thanks! Sorry to miss the vote - was AFK for a few days. I look forward to > testing it out anyhow. > > On 5 April 2018 at 14:28, Ryan Blue wrote: > > > I just sent a vote for this. Took longer than expected because I had to > > fix all of the javadoc warnings for java 8. Please test it out and vote. > > > > On Fri, Mar 30, 2018 at 10:44 AM, Ryan Blue wrote: > > > >> I have no plan for 1.9.1. > >> > >> On Fri, Mar 30, 2018 at 10:42 AM, Henry Robinson > >> wrote: > >> > >>> Great! Do you know of any plans to do a 1.9.1? > >>> > >>> On 30 March 2018 at 09:35, Ryan Blue > wrote: > >>> > I'm planning on getting a 1.10.0 rc out today, if I don't find > problems > with the stats changes. > > On Thu, Mar 29, 2018 at 4:18 PM, Henry Robinson > wrote: > > > Hi all - > > > > While using Spark, I got hit by PARQUET-1217 today on some data > written by > > Impala. This is a pretty nasty bug, and one that affects Apache > Spark > right > > now because, AFAICT, there's no release to move to that contains the > fix, > > and parquet-mr 1.9.0 is affected. There is a workaround, but it's > expensive > > in terms of lost performance. > > > > I'm new to the community, so wanted to see if there was a plan to > make a > > release (1.9.1?) in the near future. I'd rather that than have to > build > > short-term workarounds into Spark. > > > > Best, > > Henry > > > > > > -- > Ryan Blue > Software Engineer > Netflix > > >>> > >>> > >>> > >>> -- > >>> Henry Robinson > >>> Software Engineer > >>> Cloudera > >>> 415-994-6679 <(415)%20994-6679> > >>> > >> > >> > >> > >> -- > >> Ryan Blue > >> Software Engineer > >> Netflix > >> > > > > > > > > -- > > Ryan Blue > > Software Engineer > > Netflix > > >
Re: parquet-mr next release with PARQUET-1217?
Thanks! Sorry to miss the vote - was AFK for a few days. I look forward to testing it out anyhow. On 5 April 2018 at 14:28, Ryan Blue wrote: > I just sent a vote for this. Took longer than expected because I had to > fix all of the javadoc warnings for java 8. Please test it out and vote. > > On Fri, Mar 30, 2018 at 10:44 AM, Ryan Blue wrote: > >> I have no plan for 1.9.1. >> >> On Fri, Mar 30, 2018 at 10:42 AM, Henry Robinson >> wrote: >> >>> Great! Do you know of any plans to do a 1.9.1? >>> >>> On 30 March 2018 at 09:35, Ryan Blue wrote: >>> I'm planning on getting a 1.10.0 rc out today, if I don't find problems with the stats changes. On Thu, Mar 29, 2018 at 4:18 PM, Henry Robinson wrote: > Hi all - > > While using Spark, I got hit by PARQUET-1217 today on some data written by > Impala. This is a pretty nasty bug, and one that affects Apache Spark right > now because, AFAICT, there's no release to move to that contains the fix, > and parquet-mr 1.9.0 is affected. There is a workaround, but it's expensive > in terms of lost performance. > > I'm new to the community, so wanted to see if there was a plan to make a > release (1.9.1?) in the near future. I'd rather that than have to build > short-term workarounds into Spark. > > Best, > Henry > -- Ryan Blue Software Engineer Netflix >>> >>> >>> >>> -- >>> Henry Robinson >>> Software Engineer >>> Cloudera >>> 415-994-6679 <(415)%20994-6679> >>> >> >> >> >> -- >> Ryan Blue >> Software Engineer >> Netflix >> > > > > -- > Ryan Blue > Software Engineer > Netflix >
Re: parquet-mr next release with PARQUET-1217?
I just sent a vote for this. Took longer than expected because I had to fix all of the javadoc warnings for java 8. Please test it out and vote. On Fri, Mar 30, 2018 at 10:44 AM, Ryan Blue wrote: > I have no plan for 1.9.1. > > On Fri, Mar 30, 2018 at 10:42 AM, Henry Robinson > wrote: > >> Great! Do you know of any plans to do a 1.9.1? >> >> On 30 March 2018 at 09:35, Ryan Blue wrote: >> >>> I'm planning on getting a 1.10.0 rc out today, if I don't find problems >>> with the stats changes. >>> >>> On Thu, Mar 29, 2018 at 4:18 PM, Henry Robinson >>> wrote: >>> >>> > Hi all - >>> > >>> > While using Spark, I got hit by PARQUET-1217 today on some data >>> written by >>> > Impala. This is a pretty nasty bug, and one that affects Apache Spark >>> right >>> > now because, AFAICT, there's no release to move to that contains the >>> fix, >>> > and parquet-mr 1.9.0 is affected. There is a workaround, but it's >>> expensive >>> > in terms of lost performance. >>> > >>> > I'm new to the community, so wanted to see if there was a plan to make >>> a >>> > release (1.9.1?) in the near future. I'd rather that than have to build >>> > short-term workarounds into Spark. >>> > >>> > Best, >>> > Henry >>> > >>> >>> >>> >>> -- >>> Ryan Blue >>> Software Engineer >>> Netflix >>> >> >> >> >> -- >> Henry Robinson >> Software Engineer >> Cloudera >> 415-994-6679 <(415)%20994-6679> >> > > > > -- > Ryan Blue > Software Engineer > Netflix > -- Ryan Blue Software Engineer Netflix
Re: parquet-mr next release with PARQUET-1217?
I have no plan for 1.9.1. On Fri, Mar 30, 2018 at 10:42 AM, Henry Robinson wrote: > Great! Do you know of any plans to do a 1.9.1? > > On 30 March 2018 at 09:35, Ryan Blue wrote: > >> I'm planning on getting a 1.10.0 rc out today, if I don't find problems >> with the stats changes. >> >> On Thu, Mar 29, 2018 at 4:18 PM, Henry Robinson wrote: >> >> > Hi all - >> > >> > While using Spark, I got hit by PARQUET-1217 today on some data written >> by >> > Impala. This is a pretty nasty bug, and one that affects Apache Spark >> right >> > now because, AFAICT, there's no release to move to that contains the >> fix, >> > and parquet-mr 1.9.0 is affected. There is a workaround, but it's >> expensive >> > in terms of lost performance. >> > >> > I'm new to the community, so wanted to see if there was a plan to make a >> > release (1.9.1?) in the near future. I'd rather that than have to build >> > short-term workarounds into Spark. >> > >> > Best, >> > Henry >> > >> >> >> >> -- >> Ryan Blue >> Software Engineer >> Netflix >> > > > > -- > Henry Robinson > Software Engineer > Cloudera > 415-994-6679 <(415)%20994-6679> > -- Ryan Blue Software Engineer Netflix
Re: parquet-mr next release with PARQUET-1217?
Great! Do you know of any plans to do a 1.9.1? On 30 March 2018 at 09:35, Ryan Blue wrote: > I'm planning on getting a 1.10.0 rc out today, if I don't find problems > with the stats changes. > > On Thu, Mar 29, 2018 at 4:18 PM, Henry Robinson wrote: > > > Hi all - > > > > While using Spark, I got hit by PARQUET-1217 today on some data written > by > > Impala. This is a pretty nasty bug, and one that affects Apache Spark > right > > now because, AFAICT, there's no release to move to that contains the fix, > > and parquet-mr 1.9.0 is affected. There is a workaround, but it's > expensive > > in terms of lost performance. > > > > I'm new to the community, so wanted to see if there was a plan to make a > > release (1.9.1?) in the near future. I'd rather that than have to build > > short-term workarounds into Spark. > > > > Best, > > Henry > > > > > > -- > Ryan Blue > Software Engineer > Netflix > -- Henry Robinson Software Engineer Cloudera 415-994-6679
Re: parquet-mr next release with PARQUET-1217?
I'm planning on getting a 1.10.0 rc out today, if I don't find problems with the stats changes. On Thu, Mar 29, 2018 at 4:18 PM, Henry Robinson wrote: > Hi all - > > While using Spark, I got hit by PARQUET-1217 today on some data written by > Impala. This is a pretty nasty bug, and one that affects Apache Spark right > now because, AFAICT, there's no release to move to that contains the fix, > and parquet-mr 1.9.0 is affected. There is a workaround, but it's expensive > in terms of lost performance. > > I'm new to the community, so wanted to see if there was a plan to make a > release (1.9.1?) in the near future. I'd rather that than have to build > short-term workarounds into Spark. > > Best, > Henry > -- Ryan Blue Software Engineer Netflix
