IMO it's too risky to add something that adds a dependency (aws-sdk-cpp) on the day of cutting a release.
Neal On Wed, Sep 25, 2019 at 12:54 PM Krisztián Szűcs <szucs.kriszt...@gmail.com> wrote: > > We don't have a comprehensive documentation yet, so let's postpone it. > > > On Wed, Sep 25, 2019 at 9:48 PM Krisztián Szűcs <szucs.kriszt...@gmail.com> > wrote: >> >> The S3 python bindings would be a nice addition to the release. >> I don't think we should block on this but the PR is ready. Opinions? >> https://github.com/apache/arrow/pull/5423 >> >> >> >> >> On Wed, Sep 25, 2019 at 5:28 PM Micah Kornfield <emkornfi...@gmail.com> >> wrote: >>> >>> OK, I'll start the process today. I'll send up e-mail updates as I make >>> progress. >>> >>> On Wed, Sep 25, 2019 at 8:22 AM Wes McKinney <wesmck...@gmail.com> wrote: >>>> >>>> Yes, all systems go as far as I'm concerned. >>>> >>>> On Wed, Sep 25, 2019 at 9:56 AM Neal Richardson >>>> <neal.p.richard...@gmail.com> wrote: >>>> > >>>> > Andy's DataFusion issue and Wes's Parquet one have both been merged, >>>> > and it looks like the LICENSE issue is being resolved as I type. So >>>> > are we good to go now? >>>> > >>>> > Neal >>>> > >>>> > >>>> > On Tue, Sep 24, 2019 at 10:30 PM Andy Grove <andygrov...@gmail.com> >>>> > wrote: >>>> > > >>>> > > I found a last minute issue with DataFusion (Rust) and would >>>> > > appreciate it >>>> > > if we could merge ARROW-6086 (PR is >>>> > > https://github.com/apache/arrow/pull/5494) before cutting the RC. >>>> > > >>>> > > Thanks, >>>> > > >>>> > > Andy. >>>> > > >>>> > > >>>> > > On Tue, Sep 24, 2019 at 6:19 PM Micah Kornfield <emkornfi...@gmail.com> >>>> > > wrote: >>>> > > >>>> > > > OK, I'm going to postpone cutting a release until tomorrow (hoping >>>> > > > we can >>>> > > > issues resolved by then).. I'll also try to review the third-party >>>> > > > additions since 14.x. >>>> > > > >>>> > > > On Tue, Sep 24, 2019 at 4:20 PM Wes McKinney <wesmck...@gmail.com> >>>> > > > wrote: >>>> > > > >>>> > > > > I found a licensing issue >>>> > > > > >>>> > > > > https://issues.apache.org/jira/browse/ARROW-6679 >>>> > > > > >>>> > > > > It might be worth examining third party code added to the project >>>> > > > > since 0.14.x to make sure there are no other such issues. >>>> > > > > >>>> > > > > On Tue, Sep 24, 2019 at 6:10 PM Wes McKinney <wesmck...@gmail.com> >>>> > > > wrote: >>>> > > > > > >>>> > > > > > I have diagnosed the problem (Thrift "string" data must be UTF-8, >>>> > > > > > cannot be arbitrary binary) and am working on a patch right now >>>> > > > > > >>>> > > > > > On Tue, Sep 24, 2019 at 6:02 PM Wes McKinney >>>> > > > > > <wesmck...@gmail.com> >>>> > > > > wrote: >>>> > > > > > > >>>> > > > > > > I just opened >>>> > > > > > > >>>> > > > > > > https://issues.apache.org/jira/browse/ARROW-6678 >>>> > > > > > > >>>> > > > > > > Please don't cut an RC until I have an opportunity to diagnose >>>> > > > > > > this, >>>> > > > > > > will report back. >>>> > > > > > > >>>> > > > > > > >>>> > > > > > > On Tue, Sep 24, 2019 at 5:51 PM Wes McKinney >>>> > > > > > > <wesmck...@gmail.com> >>>> > > > > wrote: >>>> > > > > > > > >>>> > > > > > > > I'm investigating a possible Parquet-related compatibility >>>> > > > > > > > bug >>>> > > > that I >>>> > > > > > > > encountered through some routine testing / benchmarking. I'll >>>> > > > report >>>> > > > > > > > back once I figure out what is going on (if anything) >>>> > > > > > > > >>>> > > > > > > > On Sun, Sep 22, 2019 at 11:51 PM Micah Kornfield < >>>> > > > > emkornfi...@gmail.com> wrote: >>>> > > > > > > > >> >>>> > > > > > > > >> It's ideal if your GPG key is in the web of trust (i.e. >>>> > > > > > > > >> you can >>>> > > > > get it >>>> > > > > > > > >> signed by another PMC member), but is not 100% essential. >>>> > > > > > > > > >>>> > > > > > > > > That won't be an option for me this week (it seems like I >>>> > > > > > > > > would >>>> > > > > need to meet one face-to-face). I'll try to get the GPG checked >>>> > > > > in and >>>> > > > the >>>> > > > > rest of the pre-requisites done tomorrow (Monday) to hopefully >>>> > > > > start the >>>> > > > > release on Tuesday (hopefully we can solve the last >>>> > > > > blocker/integration >>>> > > > > tests by then). >>>> > > > > > > > > >>>> > > > > > > > > On Sat, Sep 21, 2019 at 7:12 PM Wes McKinney < >>>> > > > wesmck...@gmail.com> >>>> > > > > wrote: >>>> > > > > > > > >> >>>> > > > > > > > >> It's ideal if your GPG key is in the web of trust (i.e. >>>> > > > > > > > >> you can >>>> > > > > get it >>>> > > > > > > > >> signed by another PMC member), but is not 100% essential. >>>> > > > > > > > >> >>>> > > > > > > > >> Speaking of the release, there are at least 2 code >>>> > > > > > > > >> changes I >>>> > > > still >>>> > > > > > > > >> want to get in >>>> > > > > > > > >> >>>> > > > > > > > >> ARROW-5717 >>>> > > > > > > > >> ARROW-6353 >>>> > > > > > > > >> >>>> > > > > > > > >> I just pushed updates to ARROW-5717, will merge once the >>>> > > > > > > > >> build >>>> > > > is >>>> > > > > green. >>>> > > > > > > > >> >>>> > > > > > > > >> There are a couple of Rust patches still marked for 0.15. >>>> > > > > > > > >> The >>>> > > > rest >>>> > > > > > > > >> seems to be documentation and a couple of integration test >>>> > > > > failures we >>>> > > > > > > > >> should see about fixing in time. >>>> > > > > > > > >> >>>> > > > > > > > >> On Fri, Sep 20, 2019 at 11:26 PM Micah Kornfield < >>>> > > > > emkornfi...@gmail.com> wrote: >>>> > > > > > > > >> > >>>> > > > > > > > >> > Thanks Krisztián and Wes, >>>> > > > > > > > >> > I've gone ahead and started registering myself on all >>>> > > > > > > > >> > the >>>> > > > > packaging sites. >>>> > > > > > > > >> > >>>> > > > > > > > >> > Is there any review process when adding my GPG key to >>>> > > > > > > > >> > the SVN >>>> > > > > file? [1] >>>> > > > > > > > >> > doesn't seem to mention explicitly. >>>> > > > > > > > >> > >>>> > > > > > > > >> > Thanks, >>>> > > > > > > > >> > Micah >>>> > > > > > > > >> > >>>> > > > > > > > >> > [1] >>>> > > > > > > > >> > https://www.apache.org/dev/version-control.html#https-svn >>>> > > > > > > > >> > >>>> > > > > > > > >> > On Fri, Sep 20, 2019 at 5:01 PM Krisztián Szűcs < >>>> > > > > szucs.kriszt...@gmail.com> >>>> > > > > > > > >> > wrote: >>>> > > > > > > > >> > >>>> > > > > > > > >> > > On Thu, Sep 19, 2019 at 5:52 PM Wes McKinney < >>>> > > > > wesmck...@gmail.com> wrote: >>>> > > > > > > > >> > > >>>> > > > > > > > >> > >> On Thu, Sep 19, 2019 at 12:13 AM Micah Kornfield < >>>> > > > > emkornfi...@gmail.com> >>>> > > > > > > > >> > >> wrote: >>>> > > > > > > > >> > >> >> >>>> > > > > > > > >> > >> >> The process should be well documented at this >>>> > > > > > > > >> > >> >> point but >>>> > > > > there are a >>>> > > > > > > > >> > >> >> number of steps. >>>> > > > > > > > >> > >> > >>>> > > > > > > > >> > >> > Is [1] the up-to-date documentation for the >>>> > > > > > > > >> > >> > release? >>>> > > > Are >>>> > > > > there >>>> > > > > > > > >> > >> instructions for the adding the code signing Key to >>>> > > > > > > > >> > >> SVN? >>>> > > > > > > > >> > >> > >>>> > > > > > > > >> > >> > I will make a go of it. i will try to mitigate any >>>> > > > > internet issues by >>>> > > > > > > > >> > >> doing the process for a cloud instance (I assume >>>> > > > > > > > >> > >> that isn't >>>> > > > > a problem?). >>>> > > > > > > > >> > >> > >>>> > > > > > > > >> > >> >>>> > > > > > > > >> > >> Setting up a new cloud environment suitable for >>>> > > > > > > > >> > >> producing >>>> > > > an >>>> > > > > RC may be >>>> > > > > > > > >> > >> time consuming, but you are welcome to try. >>>> > > > > > > > >> > >> Krisztian -- >>>> > > > are >>>> > > > > you >>>> > > > > > > > >> > >> available next week to help Micah and potentially >>>> > > > > > > > >> > >> take over >>>> > > > > producing >>>> > > > > > > > >> > >> the RC if there are issues? >>>> > > > > > > > >> > >> >>>> > > > > > > > >> > > Sure, I'll be available next week. We can also grant >>>> > > > > > > > >> > > access >>>> > > > to >>>> > > > > > > > >> > > https://github.com/ursa-labs/crossbow because >>>> > > > > > > > >> > > configuring >>>> > > > all >>>> > > > > > > > >> > > the CI backends can be time consuming. >>>> > > > > > > > >> > > >>>> > > > > > > > >> > >> >>>> > > > > > > > >> > >> > Thanks, >>>> > > > > > > > >> > >> > Micah >>>> > > > > > > > >> > >> > >>>> > > > > > > > >> > >> > [1] >>>> > > > > > > > >> > >> >>>> > > > > >>>> > > > https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide >>>> > > > > > > > >> > >> > >>>> > > > > > > > >> > >> > On Wed, Sep 18, 2019 at 8:29 AM Wes McKinney < >>>> > > > > wesmck...@gmail.com> >>>> > > > > > > > >> > >> wrote: >>>> > > > > > > > >> > >> >> >>>> > > > > > > > >> > >> >> The process should be well documented at this >>>> > > > > > > > >> > >> >> point but >>>> > > > > there are a >>>> > > > > > > > >> > >> >> number of steps. Note that you need to add your >>>> > > > > > > > >> > >> >> code >>>> > > > > signing key to >>>> > > > > > > > >> > >> >> the KEYS file in SVN (that's not very hard to >>>> > > > > > > > >> > >> >> do). I >>>> > > > > think it's fine >>>> > > > > > > > >> > >> >> to hand off the process to others after the VOTE >>>> > > > > > > > >> > >> >> but it >>>> > > > > would be >>>> > > > > > > > >> > >> >> tricky to have multiple RMs involved with >>>> > > > > > > > >> > >> >> producing the >>>> > > > > source and >>>> > > > > > > > >> > >> >> binary artifacts for the vote >>>> > > > > > > > >> > >> >> >>>> > > > > > > > >> > >> >> On Tue, Sep 17, 2019 at 10:55 PM Micah Kornfield < >>>> > > > > > > > >> > >> emkornfi...@gmail.com> wrote: >>>> > > > > > > > >> > >> >> > >>>> > > > > > > > >> > >> >> > SGTM, as well. >>>> > > > > > > > >> > >> >> > >>>> > > > > > > > >> > >> >> > I should have a little bit of time next week if >>>> > > > > > > > >> > >> >> > I can >>>> > > > > help as RM but >>>> > > > > > > > >> > >> I have >>>> > > > > > > > >> > >> >> > a couple of concerns: >>>> > > > > > > > >> > >> >> > 1. In the past I've had trouble downloading and >>>> > > > > validating >>>> > > > > > > > >> > >> releases. I'm a >>>> > > > > > > > >> > >> >> > bit worried, that I might have similar problems >>>> > > > > > > > >> > >> >> > doing >>>> > > > > the necessary >>>> > > > > > > > >> > >> uploads. >>>> > > > > > > > >> > >> >> > 2. My internet connection will likely be not >>>> > > > > > > > >> > >> >> > great, I >>>> > > > > don't know if >>>> > > > > > > > >> > >> this >>>> > > > > > > > >> > >> >> > would make it even less likely to be successful. >>>> > > > > > > > >> > >> >> > >>>> > > > > > > > >> > >> >> > Does it become problematic if somehow I would >>>> > > > > > > > >> > >> >> > have to >>>> > > > > abandon the >>>> > > > > > > > >> > >> process >>>> > > > > > > > >> > >> >> > mid-release? Is there anyone who could serve >>>> > > > > > > > >> > >> >> > as a >>>> > > > > backup? Are the >>>> > > > > > > > >> > >> steps >>>> > > > > > > > >> > >> >> > well documented? >>>> > > > > > > > >> > >> >> > >>>> > > > > > > > >> > >> >> > Thanks, >>>> > > > > > > > >> > >> >> > Micah >>>> > > > > > > > >> > >> >> > >>>> > > > > > > > >> > >> >> > On Tue, Sep 17, 2019 at 4:25 PM Neal Richardson >>>> > > > > > > > >> > >> >> > < >>>> > > > > > > > >> > >> neal.p.richard...@gmail.com> >>>> > > > > > > > >> > >> >> > wrote: >>>> > > > > > > > >> > >> >> > >>>> > > > > > > > >> > >> >> > > Sounds good to me. >>>> > > > > > > > >> > >> >> > > >>>> > > > > > > > >> > >> >> > > Do we have a release manager yet? Any >>>> > > > > > > > >> > >> >> > > volunteers? >>>> > > > > > > > >> > >> >> > > >>>> > > > > > > > >> > >> >> > > Neal >>>> > > > > > > > >> > >> >> > > >>>> > > > > > > > >> > >> >> > > On Tue, Sep 17, 2019 at 4:06 PM Wes McKinney < >>>> > > > > wesmck...@gmail.com> >>>> > > > > > > > >> > >> wrote: >>>> > > > > > > > >> > >> >> > > >>>> > > > > > > > >> > >> >> > > > hi all, >>>> > > > > > > > >> > >> >> > > > >>>> > > > > > > > >> > >> >> > > > It looks like we're drawing close to be >>>> > > > > > > > >> > >> >> > > > able to >>>> > > > > make the 0.15.0 >>>> > > > > > > > >> > >> >> > > > release. I would suggest "pencils down" at >>>> > > > > > > > >> > >> >> > > > the end >>>> > > > > of this week >>>> > > > > > > > >> > >> and >>>> > > > > > > > >> > >> >> > > > see if a release candidate can be produced >>>> > > > > > > > >> > >> >> > > > next >>>> > > > > Monday September >>>> > > > > > > > >> > >> 23. >>>> > > > > > > > >> > >> >> > > > Any thoughts or objections? >>>> > > > > > > > >> > >> >> > > > >>>> > > > > > > > >> > >> >> > > > Thanks, >>>> > > > > > > > >> > >> >> > > > Wes >>>> > > > > > > > >> > >> >> > > > >>>> > > > > > > > >> > >> >> > > > On Wed, Sep 11, 2019 at 11:23 AM Wes >>>> > > > > > > > >> > >> >> > > > McKinney < >>>> > > > > > > > >> > >> wesmck...@gmail.com> >>>> > > > > > > > >> > >> >> > > wrote: >>>> > > > > > > > >> > >> >> > > > > >>>> > > > > > > > >> > >> >> > > > > hi Eric -- yes, that's correct. I'm >>>> > > > > > > > >> > >> >> > > > > planning to >>>> > > > > amend the >>>> > > > > > > > >> > >> Format docs >>>> > > > > > > > >> > >> >> > > > > today regarding the EOS issue and also >>>> > > > > > > > >> > >> >> > > > > update >>>> > > > the >>>> > > > > C++ library >>>> > > > > > > > >> > >> >> > > > > >>>> > > > > > > > >> > >> >> > > > > On Wed, Sep 11, 2019 at 11:21 AM Eric >>>> > > > > > > > >> > >> >> > > > > Erhardt >>>> > > > > > > > >> > >> >> > > > > <eric.erha...@microsoft.com> wrote: >>>> > > > > > > > >> > >> >> > > > > > >>>> > > > > > > > >> > >> >> > > > > > I assume the plan is to merge the >>>> > > > > > > > >> > >> ARROW-6313-flatbuffer-alignment >>>> > > > > > > > >> > >> >> > > > branch into master before the 0.15 release, >>>> > > > correct? >>>> > > > > > > > >> > >> >> > > > > > >>>> > > > > > > > >> > >> >> > > > > > BTW - I believe the C# alignment >>>> > > > > > > > >> > >> >> > > > > > changes are >>>> > > > > ready to be >>>> > > > > > > > >> > >> merged into >>>> > > > > > > > >> > >> >> > > > the alignment branch - >>>> > > > > > > > >> > >> https://github.com/apache/arrow/pull/5280/ >>>> > > > > > > > >> > >> >> > > > > > >>>> > > > > > > > >> > >> >> > > > > > Eric >>>> > > > > > > > >> > >> >> > > > > > >>>> > > > > > > > >> > >> >> > > > > > -----Original Message----- >>>> > > > > > > > >> > >> >> > > > > > From: Micah Kornfield >>>> > > > > > > > >> > >> >> > > > > > <emkornfi...@gmail.com> >>>> > > > > > > > >> > >> >> > > > > > Sent: Tuesday, September 10, 2019 10:24 >>>> > > > > > > > >> > >> >> > > > > > PM >>>> > > > > > > > >> > >> >> > > > > > To: Wes McKinney <wesmck...@gmail.com> >>>> > > > > > > > >> > >> >> > > > > > Cc: dev <dev@arrow.apache.org>; niki.lj >>>> > > > > > > > >> > >> >> > > > > > < >>>> > > > > niki...@aliyun.com> >>>> > > > > > > > >> > >> >> > > > > > Subject: Re: Timeline for 0.15.0 release >>>> > > > > > > > >> > >> >> > > > > > >>>> > > > > > > > >> > >> >> > > > > > I should have a little more bandwidth >>>> > > > > > > > >> > >> >> > > > > > to help >>>> > > > > with some of >>>> > > > > > > > >> > >> the >>>> > > > > > > > >> > >> >> > > > packaging starting tomorrow and going into >>>> > > > > > > > >> > >> >> > > > the >>>> > > > > weekend. >>>> > > > > > > > >> > >> >> > > > > > >>>> > > > > > > > >> > >> >> > > > > > On Tuesday, September 10, 2019, Wes >>>> > > > > > > > >> > >> >> > > > > > McKinney < >>>> > > > > > > > >> > >> wesmck...@gmail.com> >>>> > > > > > > > >> > >> >> > > > wrote: >>>> > > > > > > > >> > >> >> > > > > > >>>> > > > > > > > >> > >> >> > > > > > > Hi folks, >>>> > > > > > > > >> > >> >> > > > > > > >>>> > > > > > > > >> > >> >> > > > > > > With the state of nightly packaging >>>> > > > > > > > >> > >> >> > > > > > > and >>>> > > > > integration builds >>>> > > > > > > > >> > >> things >>>> > > > > > > > >> > >> >> > > > > > > aren't looking too good for being in >>>> > > > > > > > >> > >> >> > > > > > > release >>>> > > > > readiness by >>>> > > > > > > > >> > >> the end >>>> > > > > > > > >> > >> >> > > of >>>> > > > > > > > >> > >> >> > > > > > > this week but maybe I'm wrong. I'm >>>> > > > > > > > >> > >> >> > > > > > > planning >>>> > > > > to be working >>>> > > > > > > > >> > >> to close >>>> > > > > > > > >> > >> >> > > as >>>> > > > > > > > >> > >> >> > > > > > > many issues as I can and also to help >>>> > > > > > > > >> > >> >> > > > > > > with >>>> > > > > the ongoing >>>> > > > > > > > >> > >> alignment >>>> > > > > > > > >> > >> >> > > > fixes. >>>> > > > > > > > >> > >> >> > > > > > > >>>> > > > > > > > >> > >> >> > > > > > > Wes >>>> > > > > > > > >> > >> >> > > > > > > >>>> > > > > > > > >> > >> >> > > > > > > On Thu, Sep 5, 2019, 11:07 PM Micah >>>> > > > Kornfield >>>> > > > > < >>>> > > > > > > > >> > >> >> > > emkornfi...@gmail.com >>>> > > > > > > > >> > >> >> > > > > >>>> > > > > > > > >> > >> >> > > > > > > wrote: >>>> > > > > > > > >> > >> >> > > > > > > >>>> > > > > > > > >> > >> >> > > > > > >> Just for reference [1] has a >>>> > > > > > > > >> > >> >> > > > > > >> dashboard of >>>> > > > > the current >>>> > > > > > > > >> > >> issues: >>>> > > > > > > > >> > >> >> > > > > > >> >>>> > > > > > > > >> > >> >> > > > > > >> >>>> > > > > > > > >> > >> >> > > > >>>> > > > > > > > >> > >> >>>> > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwi >>>> > > > > > > > >> > >> >> > > > > > >> ki.apache.org >>>> > > > > > > > >> > >> >> > > > >>>> > > > > %2Fconfluence%2Fdisplay%2FARROW%2FArrow%2B0.15.0%2BRelea >>>> > > > > > > > >> > >> >> > > > > > >> se&data=02%7C01%7CEric.Erhardt% >>>> > > > > 40microsoft.com >>>> > > > > > > > >> > >> >> > > > %7Ccbead81a42104034 >>>> > > > > > > > >> > >> >> > > > > > >> >>>> > > > > > > > >> > >> >> > > > >>>> > > > > > > > >> > >> >>>> > > > > a4f308d736678a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6370376 >>>> > > > > > > > >> > >> >> > > > > > >> >>>> > > > > > > > >> > >> >> > > > >>>> > > > > > > > >> > >> >>>> > > > > 90648216338&sdata=0Upux3i%2B9X6f8uanGKSGM5VYxR6c2ADWrxSPi1%2FgbH4 >>>> > > > > > > > >> > >> >> > > > > > >> %3D&reserved=0 >>>> > > > > > > > >> > >> >> > > > > > >> >>>> > > > > > > > >> > >> >> > > > > > >> On Thu, Sep 5, 2019 at 3:43 PM Wes >>>> > > > McKinney < >>>> > > > > > > > >> > >> wesmck...@gmail.com> >>>> > > > > > > > >> > >> >> > > > wrote: >>>> > > > > > > > >> > >> >> > > > > > >> >>>> > > > > > > > >> > >> >> > > > > > >>> hi all, >>>> > > > > > > > >> > >> >> > > > > > >>> >>>> > > > > > > > >> > >> >> > > > > > >>> It doesn't seem like we're going to >>>> > > > > > > > >> > >> >> > > > > > >>> be in >>>> > > > a >>>> > > > > position to >>>> > > > > > > > >> > >> release >>>> > > > > > > > >> > >> >> > > at >>>> > > > > > > > >> > >> >> > > > > > >>> the beginning of next week. I hope >>>> > > > > > > > >> > >> >> > > > > > >>> that >>>> > > > one >>>> > > > > more week of >>>> > > > > > > > >> > >> work (or >>>> > > > > > > > >> > >> >> > > > > > >>> less) will be enough to get us >>>> > > > > > > > >> > >> >> > > > > > >>> there. >>>> > > > Aside >>>> > > > > from merging >>>> > > > > > > > >> > >> the >>>> > > > > > > > >> > >> >> > > > > > >>> alignment changes, we need to make >>>> > > > > > > > >> > >> >> > > > > > >>> sure >>>> > > > > that our >>>> > > > > > > > >> > >> packaging jobs >>>> > > > > > > > >> > >> >> > > > > > >>> required for the release candidate >>>> > > > > > > > >> > >> >> > > > > > >>> are all >>>> > > > > working. >>>> > > > > > > > >> > >> >> > > > > > >>> >>>> > > > > > > > >> > >> >> > > > > > >>> If folks could remove issues from >>>> > > > > > > > >> > >> >> > > > > > >>> the >>>> > > > > 0.15.0 backlog >>>> > > > > > > > >> > >> that they >>>> > > > > > > > >> > >> >> > > > don't >>>> > > > > > > > >> > >> >> > > > > > >>> think they will finish by end of >>>> > > > > > > > >> > >> >> > > > > > >>> next week >>>> > > > > that would >>>> > > > > > > > >> > >> help focus >>>> > > > > > > > >> > >> >> > > > > > >>> efforts (there are currently 78 >>>> > > > > > > > >> > >> >> > > > > > >>> issues in >>>> > > > > 0.15.0 still). >>>> > > > > > > > >> > >> I am >>>> > > > > > > > >> > >> >> > > > > > >>> looking to tackle a few small >>>> > > > > > > > >> > >> >> > > > > > >>> features >>>> > > > > related to >>>> > > > > > > > >> > >> dictionaries >>>> > > > > > > > >> > >> >> > > > while >>>> > > > > > > > >> > >> >> > > > > > >>> the release window is still open. >>>> > > > > > > > >> > >> >> > > > > > >>> >>>> > > > > > > > >> > >> >> > > > > > >>> - Wes >>>> > > > > > > > >> > >> >> > > > > > >>> >>>> > > > > > > > >> > >> >> > > > > > >>> On Tue, Aug 27, 2019 at 3:48 PM Wes >>>> > > > > McKinney < >>>> > > > > > > > >> > >> >> > > wesmck...@gmail.com> >>>> > > > > > > > >> > >> >> > > > > > >>> wrote: >>>> > > > > > > > >> > >> >> > > > > > >>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > hi, >>>> > > > > > > > >> > >> >> > > > > > >>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > I think we should try to release >>>> > > > > > > > >> > >> >> > > > > > >>> > the >>>> > > > week >>>> > > > > of September >>>> > > > > > > > >> > >> 9, so >>>> > > > > > > > >> > >> >> > > > > > >>> > development work should be >>>> > > > > > > > >> > >> >> > > > > > >>> > completed by >>>> > > > > end of next >>>> > > > > > > > >> > >> week. >>>> > > > > > > > >> > >> >> > > > > > >>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > Does that seem reasonable? >>>> > > > > > > > >> > >> >> > > > > > >>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > I plan to get up a patch for the >>>> > > > protocol >>>> > > > > alignment >>>> > > > > > > > >> > >> changes for >>>> > > > > > > > >> > >> >> > > > > > >>> > C++ in the next couple of days -- >>>> > > > > > > > >> > >> >> > > > > > >>> > I >>>> > > > think >>>> > > > > that getting >>>> > > > > > > > >> > >> the >>>> > > > > > > > >> > >> >> > > > > > >>> > alignment work done is the main >>>> > > > > > > > >> > >> >> > > > > > >>> > barrier >>>> > > > > to releasing. >>>> > > > > > > > >> > >> >> > > > > > >>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > Thanks >>>> > > > > > > > >> > >> >> > > > > > >>> > Wes >>>> > > > > > > > >> > >> >> > > > > > >>> > >>>> > > > > > > > >> > >> >> > > > > > >>> > On Mon, Aug 19, 2019 at 12:25 PM >>>> > > > > > > > >> > >> >> > > > > > >>> > Ji Liu >>>> > > > > > > > >> > >> >> > > > > > >>> > <niki...@aliyun.com.invalid> >>>> > > > > > > > >> > >> >> > > > > > >>> wrote: >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > Hi, Wes, on the java side, I >>>> > > > > > > > >> > >> >> > > > > > >>> > > can think >>>> > > > > of several >>>> > > > > > > > >> > >> bugs that >>>> > > > > > > > >> > >> >> > > > need >>>> > > > > > > > >> > >> >> > > > > > >>> > > to >>>> > > > > > > > >> > >> >> > > > > > >>> be fixed or reminded. >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > i. ARROW-6040: Dictionary >>>> > > > > > > > >> > >> >> > > > > > >>> > > entries are >>>> > > > > required in >>>> > > > > > > > >> > >> IPC streams >>>> > > > > > > > >> > >> >> > > > > > >>> > > even >>>> > > > > > > > >> > >> >> > > > > > >>> when empty[1] >>>> > > > > > > > >> > >> >> > > > > > >>> > > This one is under review now, >>>> > > > > > > > >> > >> >> > > > > > >>> > > however >>>> > > > > through this >>>> > > > > > > > >> > >> PR we find >>>> > > > > > > > >> > >> >> > > > > > >>> > > that >>>> > > > > > > > >> > >> >> > > > > > >>> there seems a bug in java reading >>>> > > > > > > > >> > >> >> > > > > > >>> and >>>> > > > > writing >>>> > > > > > > > >> > >> dictionaries in IPC >>>> > > > > > > > >> > >> >> > > > > > >>> which is Inconsistent with spec[2] >>>> > > > > > > > >> > >> >> > > > > > >>> since >>>> > > > it >>>> > > > > assumes all >>>> > > > > > > > >> > >> >> > > > dictionaries >>>> > > > > > > > >> > >> >> > > > > > >>> are at the start of stream (see >>>> > > > > > > > >> > >> >> > > > > > >>> details in >>>> > > > > PR comments, >>>> > > > > > > > >> > >> and this >>>> > > > > > > > >> > >> >> > > > > > >>> fix may not catch up with version >>>> > > > > > > > >> > >> >> > > > > > >>> 0.15). >>>> > > > > @Micah Kornfield >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > ii. ARROW-1875: Write 64-bit >>>> > > > > > > > >> > >> >> > > > > > >>> > > ints as >>>> > > > > strings in >>>> > > > > > > > >> > >> integration >>>> > > > > > > > >> > >> >> > > > test >>>> > > > > > > > >> > >> >> > > > > > >>> JSON files[3] >>>> > > > > > > > >> > >> >> > > > > > >>> > > Java side code already checked >>>> > > > > > > > >> > >> >> > > > > > >>> > > in, >>>> > > > other >>>> > > > > > > > >> > >> implementations >>>> > > > > > > > >> > >> >> > > seems >>>> > > > > > > > >> > >> >> > > > not. >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > iii. ARROW-6202: OutOfMemory in >>>> > > > > JdbcAdapter[4] >>>> > > > > > > > >> > >> Caused by >>>> > > > > > > > >> > >> >> > > trying >>>> > > > > > > > >> > >> >> > > > > > >>> > > to load all records in one >>>> > > > > > > > >> > >> >> > > > > > >>> > > contiguous >>>> > > > > batch, fixed >>>> > > > > > > > >> > >> >> > > > > > >>> by providing iterator API for >>>> > > > > > > > >> > >> >> > > > > > >>> iteratively >>>> > > > > reading in >>>> > > > > > > > >> > >> >> > > ARROW-6219[5]. >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > Thanks, >>>> > > > > > > > >> > >> >> > > > > > >>> > > Ji Liu >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > [1] >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F% >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > 2Fgithub.com%2Fapache%2Farrow%2Fpull%2F4960&data=02%7C01%7CE >>>> > > > > > > > >> > >> >> > > > > > >>> > > ric.Erhardt%40microsoft.com >>>> > > > > > > > >> > >> >> > > > %7Ccbead81a42104034a4f308d736678a45%7 >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637037690648216338&a >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > mp;sdata=eDF%2FAsJmVs7WjfEuNBYo%2F1TypIN44xx1TTlK6kQHZVg%3D& >>>> > > > > > > > >> > >> >> > > > > > >>> > > reserved=0 [2] >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F% >>>> > > > > > > > >> > >> >> > > > > > >>> > > 2Farrow.apache.org >>>> > > > > > > > >> > >> >> > > > %2Fdocs%2Fipc.html&data=02%7C01%7CEric.Erh >>>> > > > > > > > >> > >> >> > > > > > >>> > > ardt%40microsoft.com >>>> > > > > > > > >> > >> >> > > > %7Ccbead81a42104034a4f308d736678a45%7C72f988 >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > bf86f141af91ab2d7cd011db47%7C1%7C0%7C637037690648216338&sdat >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > a=H0pM8bVKsOyeORDhHxLlS%2BpaS%2F5meT52wxTKmNssuMk%3D&reserve >>>> > > > > > > > >> > >> >> > > > > > >>> > > d=0 [3] >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F% >>>> > > > > > > > >> > >> >> > > > > > >>> > > 2Fissues.apache.org >>>> > > > > > > > >> > >> >> > > > %2Fjira%2Fbrowse%2FARROW-1875&data=02%7C0 >>>> > > > > > > > >> > >> >> > > > > > >>> > > 1%7CEric.Erhardt%40microsoft.com >>>> > > > > > > > >> > >> >> > > > %7Ccbead81a42104034a4f308d736678 >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637037690648216 >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > 338&sdata=coTpuoEGhfjyOSBTagdlohOTX24DQZmtbWC0gYsDmkM%3D& >>>> > > > > > > > >> > >> >> > > > > > >>> > > ;reserved=0 [4] >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F% >>>> > > > > > > > >> > >> >> > > > > > >>> > > 2Fissues.apache.org >>>> > > > > > > > >> > >> >> > > > %2Fjira%2Fbrowse%2FARROW-6202%5B5&data=02 >>>> > > > > > > > >> > >> >> > > > > > >>> > > %7C01%7CEric.Erhardt%40microsoft.com >>>> > > > > > > > >> > >> >> > > > %7Ccbead81a42104034a4f308d73 >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > 6678a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63703769064 >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > 8216338&sdata=gnyUMk8cUgwc802QBLF3eAp3mznYwonlbF0qmGyzgmY%3D >>>> > > > > > > > >> > >> >> > > > > > >>> > > &reserved=0] >>>> > > > > > > > >> > >> >> > > > > > >>> >>>> > > > > > > > >> > >> >> > > > >>>> > > > > > > > >> > >> >>>> > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fis >>>> > > > > > > > >> > >> >> > > > > > >>> sues.apache.org >>>> > > > > > > > >> > >> >> > > > >>>> > > > > %2Fjira%2Fbrowse%2FARROW-6219&data=02%7C01%7CEric >>>> > > > > > > > >> > >> >> > > > > > >>> .Erhardt%40microsoft.com >>>> > > > > > > > >> > >> >> > > > %7Ccbead81a42104034a4f308d736678a45%7C72f988 >>>> > > > > > > > >> > >> >> > > > > > >>> >>>> > > > > > > > >> > >> >> > > > >>>> > > > > > > > >> > >> >>>> > > > > bf86f141af91ab2d7cd011db47%7C1%7C0%7C637037690648216338&sdata=d3 >>>> > > > > > > > >> > >> >> > > > > > >>> >>>> > > > > > > > >> > >> >>>> > > > > LF%2BTeWSprASqO%2ByE4LywlsULHGcb1Iq%2F2byHrEPkY%3D&reserved=0 >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > ---------------------------------------------------------------- >>>> > > > > > > > >> > >> >> > > > > > >>> > > -- From:Wes McKinney < >>>> > > > > wesmck...@gmail.com> Send >>>> > > > > > > > >> > >> >> > > > > > >>> > > Time:2019年8月19日(星期一) 23:03 >>>> > > > > > > > >> > >> >> > > > > > >>> > > To:dev < >>>> > > > > > > > >> > >> dev@arrow.apache.org> >>>> > > > > > > > >> > >> >> > > > > > >>> > > Subject:Re: Timeline for 0.15.0 >>>> > > > release >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > I'm going to work some on >>>> > > > > > > > >> > >> >> > > > > > >>> > > organizing >>>> > > > > the 0.15.0 >>>> > > > > > > > >> > >> backlog some >>>> > > > > > > > >> > >> >> > > > > > >>> > > this week, if anyone wants to >>>> > > > > > > > >> > >> >> > > > > > >>> > > help >>>> > > > with >>>> > > > > grooming >>>> > > > > > > > >> > >> >> > > (particularly >>>> > > > > > > > >> > >> >> > > > > > >>> > > for languages other than >>>> > > > > > > > >> > >> >> > > > > > >>> > > C++/Python >>>> > > > > where I'm >>>> > > > > > > > >> > >> focusing) that >>>> > > > > > > > >> > >> >> > > > > > >>> > > would be helpful. There have >>>> > > > > > > > >> > >> >> > > > > > >>> > > been >>>> > > > > almost 500 JIRA >>>> > > > > > > > >> > >> issues >>>> > > > > > > > >> > >> >> > > opened >>>> > > > > > > > >> > >> >> > > > > > >>> > > since the >>>> > > > > > > > >> > >> >> > > > > > >>> > > 0.14.0 release, so we should >>>> > > > > > > > >> > >> >> > > > > > >>> > > make sure >>>> > > > > to check >>>> > > > > > > > >> > >> whether >>>> > > > > > > > >> > >> >> > > there's >>>> > > > > > > > >> > >> >> > > > > > >>> > > any regressions or other >>>> > > > > > > > >> > >> >> > > > > > >>> > > serious bugs >>>> > > > > that we should >>>> > > > > > > > >> > >> try to >>>> > > > > > > > >> > >> >> > > fix >>>> > > > > > > > >> > >> >> > > > > > >>> > > for 0.15.0. >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > On Thu, Aug 15, 2019 at 6:23 PM >>>> > > > > > > > >> > >> >> > > > > > >>> > > Wes >>>> > > > > McKinney >>>> > > > > > > > >> > >> >> > > > > > >>> > > <wesmck...@gmail.com> >>>> > > > > > > > >> > >> >> > > > > > >>> wrote: >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > The Windows wheel issue in >>>> > > > > > > > >> > >> >> > > > > > >>> > > > 0.14.1 >>>> > > > > seems to be >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2 >>>> > > > > > > > >> > >> >> > > > > > >>> > > > F%2Fissues.apache.org >>>> > > > > > > > >> > >> >> > > > %2Fjira%2Fbrowse%2FARROW-6015&data=02 >>>> > > > > > > > >> > >> >> > > > > > >>> > > > %7C01%7CEric.Erhardt% >>>> > > > 40microsoft.com >>>> > > > > > > > >> > >> >> > > > %7Ccbead81a42104034a4f308d >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > 736678a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6370376 >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > 90648216338&sdata=D9lqHR16oRAFlPaIrcXq3UtW%2BLuJQW1u0Gom2u >>>> > > > > > > > >> > >> >> > > > > > >>> > > > WEWg0%3D&reserved=0 >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > I think the root cause could >>>> > > > > > > > >> > >> >> > > > > > >>> > > > be the >>>> > > > > Windows >>>> > > > > > > > >> > >> changes in >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2 >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > F%2Fgithub.com%2Fapache%2Farrow%2Fcommit%2F&data=02%7C01%7 >>>> > > > > > > > >> > >> >> > > > > > >>> > > > CEric.Erhardt%40microsoft.com >>>> > > > > > > > >> > >> >> > > > %7Ccbead81a42104034a4f308d736678a >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > 45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63703769064821 >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > 6338&sdata=iPmFB%2BncIbmvp5D31vjB4A2KyuMP%2B83Vp7%2BDiOxvl >>>> > > > > > > > >> > >> >> > > > > > >>> > > > bs%3D&reserved=0 >>>> > > > > > > > >> > >> >> > > > > > >>> 223ae744cc2a12c60cecb5db593263a03c13f85a >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > I would be appreciative if a >>>> > > > > volunteer would look >>>> > > > > > > > >> > >> into what >>>> > > > > > > > >> > >> >> > > > > > >>> > > > was >>>> > > > > > > > >> > >> >> > > > > > >>> wrong >>>> > > > > > > > >> > >> >> > > > > > >>> > > > with the 0.14.1 wheels on >>>> > > > > > > > >> > >> >> > > > > > >>> > > > Windows. >>>> > > > > Otherwise >>>> > > > > > > > >> > >> 0.15.0 Windows >>>> > > > > > > > >> > >> >> > > > > > >>> > > > wheels will be broken, too >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > The bad wheels can be found at >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2 >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > F%2Fbintray.com%2Fapache%2Farrow%2Fpython%23files%2Fpython%252 >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >>>> > > > > F0.14.1&data=02%7C01%7CEric.Erhardt% >>>> > > > > > > > >> > >> 40microsoft.com >>>> > > > > > > > >> > >> >> > > > %7Ccbea >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > d81a42104034a4f308d736678a45%7C72f988bf86f141af91ab2d7cd011db4 >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >>>> > > > > > > > >> > >> >> > > > >>>> > > > > 7%7C1%7C0%7C637037690648216338&sdata=vZzx4HNS9qp2UWhFagqfJ >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >>>> > > > > zbY%2BGzwspH1TO3wdfrbA6Y%3D&reserved=0 >>>> > > > > > > > >> > >> >> > > > > > >>> > > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > On Thu, Aug 15, 2019 at 1:28 >>>> > > > > > > > >> > >> >> > > > > > >>> > > > PM >>>> > > > > Antoine Pitrou < >>>> > > > > > > > >> > >> >> > > > > > >>> solip...@pitrou.net> wrote: >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > On Thu, 15 Aug 2019 >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > 11:17:07 -0700 >>>> > > > > Micah >>>> > > > > > > > >> > >> Kornfield >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > <emkornfi...@gmail.com> >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > wrote: >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > > In C++ they are >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > > independent, we could >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > > have >>>> > > > > 32-bit array >>>> > > > > > > > >> > >> lengths and >>>> > > > > > > > >> > >> >> > > > > > >>> variable-length >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > > types with 64-bit >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > > offsets if >>>> > > > we >>>> > > > > wanted (we >>>> > > > > > > > >> > >> just >>>> > > > > > > > >> > >> >> > > > wouldn't >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > > be >>>> > > > > > > > >> > >> >> > > > > > >>> able to >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > > have a List child with >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > > more >>>> > > > > than INT32_MAX >>>> > > > > > > > >> > >> elements). >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > I think the point is we >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > could do >>>> > > > > this in C++ >>>> > > > > > > > >> > >> but we >>>> > > > > > > > >> > >> >> > > > don't. >>>> > > > > > > > >> > >> >> > > > > > >>> I'm not sure we >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > would have introduced the >>>> > > > "Large" >>>> > > > > types if we >>>> > > > > > > > >> > >> did. >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > 64-bit offsets take twice >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > as much >>>> > > > > space as 32-bit >>>> > > > > > > > >> > >> >> > > offsets, >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > so if >>>> > > > > > > > >> > >> >> > > > > > >>> you're >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > storing lots of small-ish >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > lists or >>>> > > > > strings, >>>> > > > > > > > >> > >> 32-bit >>>> > > > > > > > >> > >> >> > > offsets >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > are preferrable. So even >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > with >>>> > > > > 64-bit array >>>> > > > > > > > >> > >> lengths from >>>> > > > > > > > >> > >> >> > > > the >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > start >>>> > > > > > > > >> > >> >> > > > > > >>> it would >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > still be beneficial to have >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > types >>>> > > > > with 32-bit >>>> > > > > > > > >> > >> offsets. >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > Going with the limited >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > address >>>> > > > > space in Java >>>> > > > > > > > >> > >> and >>>> > > > > > > > >> > >> >> > > calling >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > it a >>>> > > > > > > > >> > >> >> > > > > > >>> reference >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > implementation seems >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > suboptimal. >>>> > > > > If a consumer >>>> > > > > > > > >> > >> uses a >>>> > > > > > > > >> > >> >> > > > "Large" >>>> > > > > > > > >> > >> >> > > > > > >>> type >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > presumably it is because >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > they >>>> > > > > need the ability >>>> > > > > > > > >> > >> to store >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > more >>>> > > > > > > > >> > >> >> > > > > > >>> than INT32_MAX >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > child elements in a >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > column, >>>> > > > > otherwise it is >>>> > > > > > > > >> > >> just >>>> > > > > > > > >> > >> >> > > wasting >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > space >>>> > > > > > > > >> > >> >> > > > > > >>> [1]. >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > Probably. Though if the >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > individual >>>> > > > > elements >>>> > > > > > > > >> > >> (lists or >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > strings) >>>> > > > > > > > >> > >> >> > > > > > >>> are >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > large, not much space is >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > wasted in >>>> > > > > proportion, >>>> > > > > > > > >> > >> so it may >>>> > > > > > > > >> > >> >> > > be >>>> > > > > > > > >> > >> >> > > > > > >>> simpler in >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > such a case to always >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > create a >>>> > > > > "Large" type >>>> > > > > > > > >> > >> array. >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > [1] I suppose >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > theoretically >>>> > > > there >>>> > > > > might be some >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > performance >>>> > > > > > > > >> > >> >> > > > > > >>> benefits on >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > 64-bit architectures to >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > > using >>>> > > > the >>>> > > > > native word >>>> > > > > > > > >> > >> sizes. >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > Concretely, common 64-bit >>>> > > > > architectures don't do >>>> > > > > > > > >> > >> that, as >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > 32-bit >>>> > > > > > > > >> > >> >> > > > > > >>> is an >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > extremely common integer >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > size even >>>> > > > > in >>>> > > > > > > > >> > >> high-performance >>>> > > > > > > > >> > >> >> > > > code. >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > Regards >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > Antoine. >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > > > >>>> > > > > > > > >> > >> >> > > > > > >>> > > >>>> > > > > > > > >> > >> >> > > > > > >>> >>>> > > > > > > > >> > >> >> > > > > > >> >>>> > > > > > > > >> > >> >> > > > >>>> > > > > > > > >> > >> >> > > >>>> > > > > > > > >> > >> >>>> > > > > > > > >> > > >>>> > > > > >>>> > > >
