Got it -- thank you On Sun, Jan 17, 2021 at 3:40 PM Krisztián Szűcs <szucs.kriszt...@gmail.com> wrote:
> -1 (binding) > > We accumulated a number of blocking packaging issues (please see the > git history on the main branch for more details). > Since Kou has fixed most of those and the rust parquet patch has > merged as well, I'm going to create another release candidate > tomorrow. > > On Sun, Jan 17, 2021 at 3:37 PM Wes McKinney <wesmck...@gmail.com> wrote: > > > > All that is needed is to create a maint-3.x.x git branch. When an RC is > > accepted we will then skip the steps where we in the past have rebased > the > > master branch. > > > > On Sun, Jan 17, 2021 at 4:55 AM Andrew Lamb <al...@influxdata.com> > wrote: > > > > > I really like the idea of creating a 3.0.0 branch where we can fix > issues > > > and merge new features to master. I don't know who (if anyone) would do > > > such a thing, but I would be happy to help -- I just don't know what > to do. > > > > > > On Sat, Jan 16, 2021 at 2:24 PM Wes McKinney <wesmck...@gmail.com> > wrote: > > > > > >> I would move that we should release make any needed follow-up RC's out > > >> of a maintenance branch and let master evolve freely > > >> > > >> On Sat, Jan 16, 2021 at 12:47 PM Neville Dipale < > nevilled...@gmail.com> > > >> wrote: > > >> > > > >> > Hi Arrow devs, > > >> > > > >> > There's some bugs in the Parquet implementation which affect > reading of > > >> > data: > > >> > > > >> > - https://issues.apache.org/jira/browse/ARROW-11269, which was > opened > > >> today, > > >> > and I just saw now. > > >> > - an issue with list schema nulls from the parquet-format's logical > > >> types. > > >> > In this > > >> > case, we misinterpret the nullness of lists read from parquet-mr, > > >> > potentially leading > > >> > to incorrect data being read. > > >> > > > >> > I discovered the second bug while bashing my head trying to fix a > bug in > > >> > the > > >> > Parquet writer (sadly spent very long on it). > > >> > > > >> > Anyways, I would like to work on PRs for the above 2 bugs tonight > and > > >> > tomorrow. > > >> > > > >> > @Krisztián @Andrew Lamb <al...@influxdata.com> , would we still be > > >> able to > > >> > merge them in time? > > >> > I've also seen the offset issues in equality checks, and am going to > > >> > review/help > > >> > out with them tomorrow. > > >> > > > >> > I haven't been feeling very well this week, so I haven't been > spending > > >> much > > >> > time > > >> > working on Arrow. > > >> > > > >> > Thanks > > >> > Neville > > >> > > > >> > > > >> > On Sat, 16 Jan 2021 at 16:34, Krisztián Szűcs < > > >> szucs.kriszt...@gmail.com> > > >> > wrote: > > >> > > > >> > > On Sat, Jan 16, 2021 at 12:51 PM Andrew Lamb < > al...@influxdata.com> > > >> wrote: > > >> > > > > > >> > > > I just saw the RC0 candidate email -- thanks Krisztián. > > >> > > > > > >> > > > Does the RC0 mean that any subsequent merges to master can now > > >> proceed > > >> > > > without affecting the 3.0.0 branch? > > >> > > Technically we don't have a 3.0 release branch, but we can always > > >> create > > >> > > one. > > >> > > So yes, the merges can proceed on master. > > >> > > > > >> > > Thanks, Krisztian > > >> > > > > > >> > > > On Fri, Jan 15, 2021 at 10:22 AM Krisztián Szűcs < > > >> > > szucs.kriszt...@gmail.com> > > >> > > > wrote: > > >> > > > > > >> > > > > The spark integration test fails against spark 3.0.1 with > > >> > > > > > > >> > > > > 12:21:51.996 WARN org.apache.spark.scheduler.TaskSetManager: > Lost > > >> task > > >> > > > > 1.0 in stage 0.0 (TID 1, 5fc0f8cfe8d2, executor driver): > > >> > > > > java.lang.NoClassDefFoundError: Could not initialize class > > >> > > > > org.apache.spark.sql.util.ArrowUtils$ > > >> > > > > ... > > >> > > > > Caused by: java.lang.RuntimeException: No > DefaultAllocationManager > > >> > > > > found on classpath. Can't allocate Arrow buffers. Please > consider > > >> > > > > adding arrow-memory-netty or arrow-memory-unsafe as a > dependency. > > >> > > > > > > >> > > > > Since this change was introduced in > > >> > > > > > > >> > > > > > > >> > > > > >> > https://github.com/apache/arrow/commit/2092e18752a9c0494799493b12eb1830052217a2 > > >> > > > > which is already a part of arrow's 2.0 release, I guess this > is > > >> not a > > >> > > > > blocker (or at least the changes are required on spark's > side?). > > >> > > > > > > >> > > > > Either way, I'm going to proceed with the release. > > >> > > > > > > >> > > > > > > >> > > > > On Fri, Jan 15, 2021 at 2:53 PM Andrew Lamb < > al...@influxdata.com > > >> > > > >> > > wrote: > > >> > > > > > > > >> > > > > > That is great news Krisztián -- thank you > > >> > > > > > > > >> > > > > > On Fri, Jan 15, 2021 at 6:50 AM Krisztián Szűcs < > > >> > > > > szucs.kriszt...@gmail.com> > > >> > > > > > wrote: > > >> > > > > > > > >> > > > > > > Hi, > > >> > > > > > > > > >> > > > > > > My plan is to cut RC0 today, just want to make sure that > the > > >> spark > > >> > > > > > > integration test works with spark's latest release. > > >> > > > > > > > > >> > > > > > > Thanks, Krisztian > > >> > > > > > > > > >> > > > > > > On Fri, Jan 15, 2021 at 12:35 PM Andrew Lamb < > > >> al...@influxdata.com > > >> > > > > > >> > > > > wrote: > > >> > > > > > > > > > >> > > > > > > > Hi, > > >> > > > > > > > > > >> > > > > > > > I apologize if I have missed this detail on previous > > >> emails; I > > >> > > > > wonder if > > >> > > > > > > > there is any estimate of when the Arrow 3.0 release > might be > > >> > > > > finalized. > > >> > > > > > > > > > >> > > > > > > > The Rust implementation has a few PRs we have been > holding > > >> off > > >> > > > > merging > > >> > > > > > > > until the release goes out and I wanted to know if there > > >> was any > > >> > > > > > > estimated > > >> > > > > > > > timeline. > > >> > > > > > > > > > >> > > > > > > > The wiki shows no blocking JIRA items (nice work > everyone!) > > >> any > > >> > > > > longer: > > >> > > > > > > > > > >> > > > > > > >> https://cwiki.apache.org/confluence/display/ARROW/Arrow+3.0.0+Release > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > Thanks, > > >> > > > > > > > Andrew > > >> > > > > > > > > >> > > > > > > >> > > > > >> > > > >
