The process should be well documented at this point but there are a number of steps. Note that you need to add your code signing key to the KEYS file in SVN (that's not very hard to do). I think it's fine to hand off the process to others after the VOTE but it would be tricky to have multiple RMs involved with producing the source and binary artifacts for the vote
On Tue, Sep 17, 2019 at 10:55 PM Micah Kornfield <emkornfi...@gmail.com> wrote: > > SGTM, as well. > > I should have a little bit of time next week if I can help as RM but I have > a couple of concerns: > 1. In the past I've had trouble downloading and validating releases. I'm a > bit worried, that I might have similar problems doing the necessary uploads. > 2. My internet connection will likely be not great, I don't know if this > would make it even less likely to be successful. > > Does it become problematic if somehow I would have to abandon the process > mid-release? Is there anyone who could serve as a backup? Are the steps > well documented? > > Thanks, > Micah > > On Tue, Sep 17, 2019 at 4:25 PM Neal Richardson <neal.p.richard...@gmail.com> > wrote: > > > Sounds good to me. > > > > Do we have a release manager yet? Any volunteers? > > > > Neal > > > > On Tue, Sep 17, 2019 at 4:06 PM Wes McKinney <wesmck...@gmail.com> wrote: > > > > > hi all, > > > > > > It looks like we're drawing close to be able to make the 0.15.0 > > > release. I would suggest "pencils down" at the end of this week and > > > see if a release candidate can be produced next Monday September 23. > > > Any thoughts or objections? > > > > > > Thanks, > > > Wes > > > > > > On Wed, Sep 11, 2019 at 11:23 AM Wes McKinney <wesmck...@gmail.com> > > wrote: > > > > > > > > hi Eric -- yes, that's correct. I'm planning to amend the Format docs > > > > today regarding the EOS issue and also update the C++ library > > > > > > > > On Wed, Sep 11, 2019 at 11:21 AM Eric Erhardt > > > > <eric.erha...@microsoft.com> wrote: > > > > > > > > > > I assume the plan is to merge the ARROW-6313-flatbuffer-alignment > > > branch into master before the 0.15 release, correct? > > > > > > > > > > BTW - I believe the C# alignment changes are ready to be merged into > > > the alignment branch - https://github.com/apache/arrow/pull/5280/ > > > > > > > > > > Eric > > > > > > > > > > -----Original Message----- > > > > > From: Micah Kornfield <emkornfi...@gmail.com> > > > > > Sent: Tuesday, September 10, 2019 10:24 PM > > > > > To: Wes McKinney <wesmck...@gmail.com> > > > > > Cc: dev <dev@arrow.apache.org>; niki.lj <niki...@aliyun.com> > > > > > Subject: Re: Timeline for 0.15.0 release > > > > > > > > > > I should have a little more bandwidth to help with some of the > > > packaging starting tomorrow and going into the weekend. > > > > > > > > > > On Tuesday, September 10, 2019, Wes McKinney <wesmck...@gmail.com> > > > wrote: > > > > > > > > > > > Hi folks, > > > > > > > > > > > > With the state of nightly packaging and integration builds things > > > > > > aren't looking too good for being in release readiness by the end > > of > > > > > > this week but maybe I'm wrong. I'm planning to be working to close > > as > > > > > > many issues as I can and also to help with the ongoing alignment > > > fixes. > > > > > > > > > > > > Wes > > > > > > > > > > > > On Thu, Sep 5, 2019, 11:07 PM Micah Kornfield < > > emkornfi...@gmail.com > > > > > > > > > > wrote: > > > > > > > > > > > >> Just for reference [1] has a dashboard of the current issues: > > > > > >> > > > > > >> > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwi > > > > > >> ki.apache.org > > > %2Fconfluence%2Fdisplay%2FARROW%2FArrow%2B0.15.0%2BRelea > > > > > >> se&data=02%7C01%7CEric.Erhardt%40microsoft.com > > > %7Ccbead81a42104034 > > > > > >> > > > a4f308d736678a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6370376 > > > > > >> > > > 90648216338&sdata=0Upux3i%2B9X6f8uanGKSGM5VYxR6c2ADWrxSPi1%2FgbH4 > > > > > >> %3D&reserved=0 > > > > > >> > > > > > >> On Thu, Sep 5, 2019 at 3:43 PM Wes McKinney <wesmck...@gmail.com> > > > wrote: > > > > > >> > > > > > >>> hi all, > > > > > >>> > > > > > >>> It doesn't seem like we're going to be in a position to release > > at > > > > > >>> the beginning of next week. I hope that one more week of work (or > > > > > >>> less) will be enough to get us there. Aside from merging the > > > > > >>> alignment changes, we need to make sure that our packaging jobs > > > > > >>> required for the release candidate are all working. > > > > > >>> > > > > > >>> If folks could remove issues from the 0.15.0 backlog that they > > > don't > > > > > >>> think they will finish by end of next week that would help focus > > > > > >>> efforts (there are currently 78 issues in 0.15.0 still). I am > > > > > >>> looking to tackle a few small features related to dictionaries > > > while > > > > > >>> the release window is still open. > > > > > >>> > > > > > >>> - Wes > > > > > >>> > > > > > >>> On Tue, Aug 27, 2019 at 3:48 PM Wes McKinney < > > wesmck...@gmail.com> > > > > > >>> wrote: > > > > > >>> > > > > > > >>> > hi, > > > > > >>> > > > > > > >>> > I think we should try to release the week of September 9, so > > > > > >>> > development work should be completed by end of next week. > > > > > >>> > > > > > > >>> > Does that seem reasonable? > > > > > >>> > > > > > > >>> > I plan to get up a patch for the protocol alignment changes for > > > > > >>> > C++ in the next couple of days -- I think that getting the > > > > > >>> > alignment work done is the main barrier to releasing. > > > > > >>> > > > > > > >>> > Thanks > > > > > >>> > Wes > > > > > >>> > > > > > > >>> > On Mon, Aug 19, 2019 at 12:25 PM Ji Liu > > > > > >>> > <niki...@aliyun.com.invalid> > > > > > >>> wrote: > > > > > >>> > > > > > > > >>> > > Hi, Wes, on the java side, I can think of several bugs that > > > need > > > > > >>> > > to > > > > > >>> be fixed or reminded. > > > > > >>> > > > > > > > >>> > > i. ARROW-6040: Dictionary entries are required in IPC streams > > > > > >>> > > even > > > > > >>> when empty[1] > > > > > >>> > > This one is under review now, however through this PR we find > > > > > >>> > > that > > > > > >>> there seems a bug in java reading and writing dictionaries in IPC > > > > > >>> which is Inconsistent with spec[2] since it assumes all > > > dictionaries > > > > > >>> are at the start of stream (see details in PR comments, and this > > > > > >>> fix may not catch up with version 0.15). @Micah Kornfield > > > > > >>> > > > > > > > >>> > > ii. ARROW-1875: Write 64-bit ints as strings in integration > > > test > > > > > >>> JSON files[3] > > > > > >>> > > Java side code already checked in, other implementations > > seems > > > not. > > > > > >>> > > > > > > > >>> > > iii. ARROW-6202: OutOfMemory in JdbcAdapter[4] Caused by > > trying > > > > > >>> > > to load all records in one contiguous batch, fixed > > > > > >>> by providing iterator API for iteratively reading in > > ARROW-6219[5]. > > > > > >>> > > > > > > > >>> > > Thanks, > > > > > >>> > > Ji Liu > > > > > >>> > > > > > > > >>> > > [1] > > > > > >>> > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F% > > > > > >>> > > > > > 2Fgithub.com%2Fapache%2Farrow%2Fpull%2F4960&data=02%7C01%7CE > > > > > >>> > > ric.Erhardt%40microsoft.com > > > %7Ccbead81a42104034a4f308d736678a45%7 > > > > > >>> > > > > > C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637037690648216338&a > > > > > >>> > > > > > mp;sdata=eDF%2FAsJmVs7WjfEuNBYo%2F1TypIN44xx1TTlK6kQHZVg%3D& > > > > > >>> > > reserved=0 [2] > > > > > >>> > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F% > > > > > >>> > > 2Farrow.apache.org > > > %2Fdocs%2Fipc.html&data=02%7C01%7CEric.Erh > > > > > >>> > > ardt%40microsoft.com > > > %7Ccbead81a42104034a4f308d736678a45%7C72f988 > > > > > >>> > > > > > bf86f141af91ab2d7cd011db47%7C1%7C0%7C637037690648216338&sdat > > > > > >>> > > > > > a=H0pM8bVKsOyeORDhHxLlS%2BpaS%2F5meT52wxTKmNssuMk%3D&reserve > > > > > >>> > > d=0 [3] > > > > > >>> > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F% > > > > > >>> > > 2Fissues.apache.org > > > %2Fjira%2Fbrowse%2FARROW-1875&data=02%7C0 > > > > > >>> > > 1%7CEric.Erhardt%40microsoft.com > > > %7Ccbead81a42104034a4f308d736678 > > > > > >>> > > > > > a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637037690648216 > > > > > >>> > > > > > 338&sdata=coTpuoEGhfjyOSBTagdlohOTX24DQZmtbWC0gYsDmkM%3D& > > > > > >>> > > ;reserved=0 [4] > > > > > >>> > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F% > > > > > >>> > > 2Fissues.apache.org > > > %2Fjira%2Fbrowse%2FARROW-6202%5B5&data=02 > > > > > >>> > > %7C01%7CEric.Erhardt%40microsoft.com > > > %7Ccbead81a42104034a4f308d73 > > > > > >>> > > > > > 6678a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63703769064 > > > > > >>> > > > > > 8216338&sdata=gnyUMk8cUgwc802QBLF3eAp3mznYwonlbF0qmGyzgmY%3D > > > > > >>> > > &reserved=0] > > > > > >>> > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fis > > > > > >>> sues.apache.org > > > %2Fjira%2Fbrowse%2FARROW-6219&data=02%7C01%7CEric > > > > > >>> .Erhardt%40microsoft.com > > > %7Ccbead81a42104034a4f308d736678a45%7C72f988 > > > > > >>> > > > bf86f141af91ab2d7cd011db47%7C1%7C0%7C637037690648216338&sdata=d3 > > > > > >>> LF%2BTeWSprASqO%2ByE4LywlsULHGcb1Iq%2F2byHrEPkY%3D&reserved=0 > > > > > >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> > > > > > ---------------------------------------------------------------- > > > > > >>> > > -- From:Wes McKinney <wesmck...@gmail.com> Send > > > > > >>> > > Time:2019年8月19日(星期一) 23:03 To:dev <dev@arrow.apache.org> > > > > > >>> > > Subject:Re: Timeline for 0.15.0 release > > > > > >>> > > > > > > > >>> > > I'm going to work some on organizing the 0.15.0 backlog some > > > > > >>> > > this week, if anyone wants to help with grooming > > (particularly > > > > > >>> > > for languages other than C++/Python where I'm focusing) that > > > > > >>> > > would be helpful. There have been almost 500 JIRA issues > > opened > > > > > >>> > > since the > > > > > >>> > > 0.14.0 release, so we should make sure to check whether > > there's > > > > > >>> > > any regressions or other serious bugs that we should try to > > fix > > > > > >>> > > for 0.15.0. > > > > > >>> > > > > > > > >>> > > On Thu, Aug 15, 2019 at 6:23 PM Wes McKinney > > > > > >>> > > <wesmck...@gmail.com> > > > > > >>> wrote: > > > > > >>> > > > > > > > > >>> > > > The Windows wheel issue in 0.14.1 seems to be > > > > > >>> > > > > > > > > >>> > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2 > > > > > >>> > > > F%2Fissues.apache.org > > > %2Fjira%2Fbrowse%2FARROW-6015&data=02 > > > > > >>> > > > %7C01%7CEric.Erhardt%40microsoft.com > > > %7Ccbead81a42104034a4f308d > > > > > >>> > > > > > > 736678a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6370376 > > > > > >>> > > > > > > 90648216338&sdata=D9lqHR16oRAFlPaIrcXq3UtW%2BLuJQW1u0Gom2u > > > > > >>> > > > WEWg0%3D&reserved=0 > > > > > >>> > > > > > > > > >>> > > > I think the root cause could be the Windows changes in > > > > > >>> > > > > > > > > >>> > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2 > > > > > >>> > > > > > > F%2Fgithub.com%2Fapache%2Farrow%2Fcommit%2F&data=02%7C01%7 > > > > > >>> > > > CEric.Erhardt%40microsoft.com > > > %7Ccbead81a42104034a4f308d736678a > > > > > >>> > > > > > > 45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63703769064821 > > > > > >>> > > > > > > 6338&sdata=iPmFB%2BncIbmvp5D31vjB4A2KyuMP%2B83Vp7%2BDiOxvl > > > > > >>> > > > bs%3D&reserved=0 > > > > > >>> 223ae744cc2a12c60cecb5db593263a03c13f85a > > > > > >>> > > > > > > > > >>> > > > I would be appreciative if a volunteer would look into what > > > > > >>> > > > was > > > > > >>> wrong > > > > > >>> > > > with the 0.14.1 wheels on Windows. Otherwise 0.15.0 Windows > > > > > >>> > > > wheels will be broken, too > > > > > >>> > > > > > > > > >>> > > > The bad wheels can be found at > > > > > >>> > > > > > > > > >>> > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2 > > > > > >>> > > > > > > F%2Fbintray.com%2Fapache%2Farrow%2Fpython%23files%2Fpython%252 > > > > > >>> > > > F0.14.1&data=02%7C01%7CEric.Erhardt%40microsoft.com > > > %7Ccbea > > > > > >>> > > > > > > d81a42104034a4f308d736678a45%7C72f988bf86f141af91ab2d7cd011db4 > > > > > >>> > > > > > > 7%7C1%7C0%7C637037690648216338&sdata=vZzx4HNS9qp2UWhFagqfJ > > > > > >>> > > > zbY%2BGzwspH1TO3wdfrbA6Y%3D&reserved=0 > > > > > >>> > > > > > > > > >>> > > > On Thu, Aug 15, 2019 at 1:28 PM Antoine Pitrou < > > > > > >>> solip...@pitrou.net> wrote: > > > > > >>> > > > > > > > > > >>> > > > > On Thu, 15 Aug 2019 11:17:07 -0700 Micah Kornfield > > > > > >>> > > > > <emkornfi...@gmail.com> wrote: > > > > > >>> > > > > > > > > > > > >>> > > > > > > In C++ they are > > > > > >>> > > > > > > independent, we could have 32-bit array lengths and > > > > > >>> variable-length > > > > > >>> > > > > > > types with 64-bit offsets if we wanted (we just > > > wouldn't > > > > > >>> > > > > > > be > > > > > >>> able to > > > > > >>> > > > > > > have a List child with more than INT32_MAX elements). > > > > > >>> > > > > > > > > > > >>> > > > > > I think the point is we could do this in C++ but we > > > don't. > > > > > >>> I'm not sure we > > > > > >>> > > > > > would have introduced the "Large" types if we did. > > > > > >>> > > > > > > > > > >>> > > > > 64-bit offsets take twice as much space as 32-bit > > offsets, > > > > > >>> > > > > so if > > > > > >>> you're > > > > > >>> > > > > storing lots of small-ish lists or strings, 32-bit > > offsets > > > > > >>> > > > > are preferrable. So even with 64-bit array lengths from > > > the > > > > > >>> > > > > start > > > > > >>> it would > > > > > >>> > > > > still be beneficial to have types with 32-bit offsets. > > > > > >>> > > > > > > > > > >>> > > > > > Going with the limited address space in Java and > > calling > > > > > >>> > > > > > it a > > > > > >>> reference > > > > > >>> > > > > > implementation seems suboptimal. If a consumer uses a > > > "Large" > > > > > >>> type > > > > > >>> > > > > > presumably it is because they need the ability to store > > > > > >>> > > > > > more > > > > > >>> than INT32_MAX > > > > > >>> > > > > > child elements in a column, otherwise it is just > > wasting > > > > > >>> > > > > > space > > > > > >>> [1]. > > > > > >>> > > > > > > > > > >>> > > > > Probably. Though if the individual elements (lists or > > > > > >>> > > > > strings) > > > > > >>> are > > > > > >>> > > > > large, not much space is wasted in proportion, so it may > > be > > > > > >>> simpler in > > > > > >>> > > > > such a case to always create a "Large" type array. > > > > > >>> > > > > > > > > > >>> > > > > > [1] I suppose theoretically there might be some > > > > > >>> > > > > > performance > > > > > >>> benefits on > > > > > >>> > > > > > 64-bit architectures to using the native word sizes. > > > > > >>> > > > > > > > > > >>> > > > > Concretely, common 64-bit architectures don't do that, as > > > > > >>> > > > > 32-bit > > > > > >>> is an > > > > > >>> > > > > extremely common integer size even in high-performance > > > code. > > > > > >>> > > > > > > > > > >>> > > > > Regards > > > > > >>> > > > > > > > > > >>> > > > > Antoine. > > > > > >>> > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > >>> > > > > > >> > > > > >
