Re: Timeline for 0.15.0 release

2019-09-26 Thread Krisztián Szűcs
gt;> The process should be well documented at
>> > this point but
>> > >>>> > > > > there are a
>> > >>>> > > > > > > > >> > >> >> number of steps.
>> > >>>> > > > > > > > >> > >> >
>> > >>>> > > > > > > > >> > >> > Is [1] the up-to-date documentation for the
>> > release?
>> > >>>> > > >  Are
>> > >>>> > > > > there
>> > >>>> > > > > > > > >> > >> instructions for the adding the code signing
>> > Key to SVN?
>> > >>>> > > > > > > > >> > >> >
>> > >>>> > > > > > > > >> > >> > I will make a go of it.  i will try to
>> > mitigate any
>> > >>>> > > > > internet issues by
>> > >>>> > > > > > > > >> > >> doing the process for a cloud instance (I
>> > assume that isn't
>> > >>>> > > > > a problem?).
>> > >>>> > > > > > > > >> > >> >
>> > >>>> > > > > > > > >> > >>
>>
>> > >>>> > > > > > > > >> > >> Setting up a new cloud environment suitable for
>> > producing
>> > >>>> > > > an
>> > >>>> > > > > RC may be
>> > >>>> > > > > > > > >> > >> time consuming, but you are welcome to try.
>> > Krisztian --
>> > >>>> > > > are
>> > >>>> > > > > you
>> > >>>> > > > > > > > >> > >> available next week to help Micah and
>> > potentially take over
>> > >>>> > > > > producing
>> > >>>> > > > > > > > >> > >> the RC if there are issues?
>> > >>>> > > > > > > > >> > >>
>> > >>>> > > > > > > > >> > > Sure, I'll be available next week. We can also
>> > grant access
>> > >>>> > > > to
>> > >>>> > > > > > > > >> > > https://github.com/ursa-labs/crossbow because
>> > configuring
>> > >>>> > > > all
>> > >>>> > > > > > > > >> > > the CI backends can be time consuming.
>> > >>>> > > > > > > > >> > >
>> > >>>> > > > > > > > >> > >>
>> > >>>> > > > > > > > >> > >> > Thanks,
>> > >>>> > > > > > > > >> > >> > Micah
>> > >>>> > > > > > > > >> > >> >
>> > >>>> > > > > > > > >> > >> > [1]
>> > >>>> > > > > > > > >> > >>
>> > >>>> > > > >
>> > >>>> > > >
>> >
>> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
>> > >>>> > > > > > > > >> > >> >
>>
>> > >>>> > > > > > > > >> > >> > On Wed, Sep 18, 2019 at 8:29 AM Wes McKinney <
>> > >>>> > > > > wesmck...@gmail.com>
>> > >>>> > > > > > > > >> > >> wrote:
>> > >>>> > > > > > > > >> > >> >>
>> > >>>> > > > > > > > >> > >> >> The process should be well documented at
>> > this point but
>> > >>>> > > > > there are a
>> > >>>> > > > > > > > >> > >> >> number of steps. Note that you need to add
>> > your code
>> > >>>> > > > > signing key to
>> > >>>> > > > > > > > >> &g

Re: Timeline for 0.15.0 release

2019-09-26 Thread Ji Liu
> > > > > > > >> > >> > Micah
> >>>> > > > > > > > >> > >> >
> >>>> > > > > > > > >> > >> > [1]
> >>>> > > > > > > > >> > >>
> >>>> > > > >
> >>>> > > >
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
> >>>> > > > > > > > >> > >> >
> >>>> > > > > > > > >> > >> > On Wed, Sep 18, 2019 at 8:29 AM Wes McKinney <
> >>>> > > > > wesmck...@gmail.com>
> >>>> > > > > > > > >> > >> wrote:
> >>>> > > > > > > > >> > >> >>
> >>>> > > > > > > > >> > >> >> The process should be well documented at
> this point but
> >>>> > > > > there are a
> >>>> > > > > > > > >> > >> >> number of steps. Note that you need to add
> your code
> >>>> > > > > signing key to
> >>>> > > > > > > > >> > >> >> the KEYS file in SVN (that's not very hard
> to do). I
> >>>> > > > > think it's fine
> >>>> > > > > > > > >> > >> >> to hand off the process to others after the
> VOTE but it
> >>>> > > > > would be
> >>>> > > > > > > > >> > >> >> tricky to have multiple RMs involved with
> producing the
> >>>> > > > > source and
> >>>> > > > > > > > >> > >> >> binary artifacts for the vote
> >>>> > > > > > > > >> > >> >>
> >>>> > > > > > > > >> > >> >> On Tue, Sep 17, 2019 at 10:55 PM Micah
> Kornfield <
> >>>> > > > > > > > >> > >> emkornfi...@gmail.com> wrote:
> >>>> > > > > > > > >> > >> >> >
> >>>> > > > > > > > >> > >> >> > SGTM, as well.
> >>>> > > > > > > > >> > >> >> >
> >>>> > > > > > > > >> > >> >> > I should have a little bit of time next
> week if I can
> >>>> > > > > help as RM but
> >>>> > > > > > > > >> > >> I have
> >>>> > > > > > > > >> > >> >> > a couple of concerns:
> >>>> > > > > > > > >> > >> >> > 1.  In the past I've had trouble
> downloading and
> >>>> > > > > validating
> >>>> > > > > > > > >> > >> releases. I'm a
> >>>> > > > > > > > >> > >> >> > bit worried, that I might have similar
> problems doing
> >>>> > > > > the necessary
> >>>> > > > > > > > >> > >> uploads.
> >>>> > > > > > > > >> > >> >> > 2.  My internet connection will likely be
> not great, I
> >>>> > > > > don't know if
> >>>> > > > > > > > >> > >> this
> >>>> > > > > > > > >> > >> >> > would make it even less likely to be
> successful.
> >>>> > > > > > > > >> > >> >> >
> >>>> > > > > > > > >> > >> >> > Does it become problematic if somehow I
> would have to
> >>>> > > > > abandon the
> >>>> > > > > > > > >> > >> process
> >>>> > > > > > > > >> > >> >> > mid-release?  Is there anyone who could
> serve as a
> >>>> > > > > backup?  Are the
> >>>> > > > > > > > >> > >> steps
> >>>> > > > > > > > >> > >> >> > well documented?
> >>>> > > > >

Re: Timeline for 0.15.0 release

2019-09-26 Thread Micah Kornfield
doesn't seem to mention explicitly.
> >>>> > > > > > > > >> >
> >>>> > > > > > > > >> > Thanks,
> >>>> > > > > > > > >> > Micah
> >>>> > > > > > > > >> >
> >>>> > > > > > > > >> > [1]
> https://www.apache.org/dev/version-control.html#https-svn
> >>>> > > > > > > > >> >
> >>>> > > > > > > > >> > On Fri, Sep 20, 2019 at 5:01 PM Krisztián Szűcs <
> >>>> > > > > szucs.kriszt...@gmail.com>
> >>>> > > > > > > > >> > wrote:
> >>>> > > > > > > > >> >
> >>>> > > > > > > > >> > > On Thu, Sep 19, 2019 at 5:52 PM Wes McKinney <
> >>>> > > > > wesmck...@gmail.com> wrote:
> >>>> > > > > > > > >> > >
> >>>> > > > > > > > >> > >> On Thu, Sep 19, 2019 at 12:13 AM Micah
> Kornfield <
> >>>> > > > > emkornfi...@gmail.com>
> >>>> > > > > > > > >> > >> wrote:
> >>>> > > > > > > > >> > >> >>
> >>>> > > > > > > > >> > >> >> The process should be well documented at
> this point but
> >>>> > > > > there are a
> >>>> > > > > > > > >> > >> >> number of steps.
> >>>> > > > > > > > >> > >> >
> >>>> > > > > > > > >> > >> > Is [1] the up-to-date documentation for the
> release?
> >>>> > > >  Are
> >>>> > > > > there
> >>>> > > > > > > > >> > >> instructions for the adding the code signing
> Key to SVN?
> >>>> > > > > > > > >> > >> >
> >>>> > > > > > > > >> > >> > I will make a go of it.  i will try to
> mitigate any
> >>>> > > > > internet issues by
> >>>> > > > > > > > >> > >> doing the process for a cloud instance (I
> assume that isn't
> >>>> > > > > a problem?).
> >>>> > > > > > > > >> > >> >
> >>>> > > > > > > > >> > >>
> >>>> > > > > > > > >> > >> Setting up a new cloud environment suitable for
> producing
> >>>> > > > an
> >>>> > > > > RC may be
> >>>> > > > > > > > >> > >> time consuming, but you are welcome to try.
> Krisztian --
> >>>> > > > are
> >>>> > > > > you
> >>>> > > > > > > > >> > >> available next week to help Micah and
> potentially take over
> >>>> > > > > producing
> >>>> > > > > > > > >> > >> the RC if there are issues?
> >>>> > > > > > > > >> > >>
> >>>> > > > > > > > >> > > Sure, I'll be available next week. We can also
> grant access
> >>>> > > > to
> >>>> > > > > > > > >> > > https://github.com/ursa-labs/crossbow because
> configuring
> >>>> > > > all
> >>>> > > > > > > > >> > > the CI backends can be time consuming.
> >>>> > > > > > > > >> > >
> >>>> > > > > > > > >> > >>
> >>>> > > > > > > > >> > >> > Thanks,
> >>>> > > > > > > > >> > >> > Micah
> >>>> > > > > > > > >> > >> >
> >>>> > > > > > > > >> > >> > [1]
> >>>> > > > > > > > >> > >>
> >>>> > > > >
> >>>> > > &g

Re: Timeline for 0.15.0 release

2019-09-25 Thread Neal Richardson
g/dev/version-control.html#https-svn
>>>> > > > > > > > >> >
>>>> > > > > > > > >> > On Fri, Sep 20, 2019 at 5:01 PM Krisztián Szűcs <
>>>> > > > > szucs.kriszt...@gmail.com>
>>>> > > > > > > > >> > wrote:
>>>> > > > > > > > >> >
>>>> > > > > > > > >> > > On Thu, Sep 19, 2019 at 5:52 PM Wes McKinney <
>>>> > > > > wesmck...@gmail.com> wrote:
>>>> > > > > > > > >> > >
>>>> > > > > > > > >> > >> On Thu, Sep 19, 2019 at 12:13 AM Micah Kornfield <
>>>> > > > > emkornfi...@gmail.com>
>>>> > > > > > > > >> > >> wrote:
>>>> > > > > > > > >> > >> >>
>>>> > > > > > > > >> > >> >> The process should be well documented at this 
>>>> > > > > > > > >> > >> >> point but
>>>> > > > > there are a
>>>> > > > > > > > >> > >> >> number of steps.
>>>> > > > > > > > >> > >> >
>>>> > > > > > > > >> > >> > Is [1] the up-to-date documentation for the 
>>>> > > > > > > > >> > >> > release?
>>>> > > >  Are
>>>> > > > > there
>>>> > > > > > > > >> > >> instructions for the adding the code signing Key to 
>>>> > > > > > > > >> > >> SVN?
>>>> > > > > > > > >> > >> >
>>>> > > > > > > > >> > >> > I will make a go of it.  i will try to mitigate any
>>>> > > > > internet issues by
>>>> > > > > > > > >> > >> doing the process for a cloud instance (I assume 
>>>> > > > > > > > >> > >> that isn't
>>>> > > > > a problem?).
>>>> > > > > > > > >> > >> >
>>>> > > > > > > > >> > >>
>>>> > > > > > > > >> > >> Setting up a new cloud environment suitable for 
>>>> > > > > > > > >> > >> producing
>>>> > > > an
>>>> > > > > RC may be
>>>> > > > > > > > >> > >> time consuming, but you are welcome to try. 
>>>> > > > > > > > >> > >> Krisztian --
>>>> > > > are
>>>> > > > > you
>>>> > > > > > > > >> > >> available next week to help Micah and potentially 
>>>> > > > > > > > >> > >> take over
>>>> > > > > producing
>>>> > > > > > > > >> > >> the RC if there are issues?
>>>> > > > > > > > >> > >>
>>>> > > > > > > > >> > > Sure, I'll be available next week. We can also grant 
>>>> > > > > > > > >> > > access
>>>> > > > to
>>>> > > > > > > > >> > > https://github.com/ursa-labs/crossbow because 
>>>> > > > > > > > >> > > configuring
>>>> > > > all
>>>> > > > > > > > >> > > the CI backends can be time consuming.
>>>> > > > > > > > >> > >
>>>> > > > > > > > >> > >>
>>>> > > > > > > > >> > >> > Thanks,
>>>> > > > > > > > >> > >> > Micah
>>>> > > > > > > > >> > >> >
>>>> > > > > > > > >> > >> > [1]
>>>> > > > > > > > >> > >>
>>>> > > > >
>>>> > > > https://cwiki.apache.org/confluence/display/ARROW/Rel

Re: Timeline for 0.15.0 release

2019-09-25 Thread Krisztián Szűcs
; >
>> > > > > > > > >> > >>
>> > > > > > > > >> > >> Setting up a new cloud environment suitable for
>> producing
>> > > > an
>> > > > > RC may be
>> > > > > > > > >> > >> time consuming, but you are welcome to try.
>> Krisztian --
>> > > > are
>> > > > > you
>> > > > > > > > >> > >> available next week to help Micah and potentially
>> take over
>> > > > > producing
>> > > > > > > > >> > >> the RC if there are issues?
>> > > > > > > > >> > >>
>> > > > > > > > >> > > Sure, I'll be available next week. We can also grant
>> access
>> > > > to
>> > > > > > > > >> > > https://github.com/ursa-labs/crossbow because
>> configuring
>> > > > all
>> > > > > > > > >> > > the CI backends can be time consuming.
>> > > > > > > > >> > >
>> > > > > > > > >> > >>
>> > > > > > > > >> > >> > Thanks,
>> > > > > > > > >> > >> > Micah
>> > > > > > > > >> > >> >
>> > > > > > > > >> > >> > [1]
>> > > > > > > > >> > >>
>> > > > >
>> > > >
>> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
>> > > > > > > > >> > >> >
>> > > > > > > > >> > >> > On Wed, Sep 18, 2019 at 8:29 AM Wes McKinney <
>> > > > > wesmck...@gmail.com>
>> > > > > > > > >> > >> wrote:
>> > > > > > > > >> > >> >>
>> > > > > > > > >> > >> >> The process should be well documented at this
>> point but
>> > > > > there are a
>> > > > > > > > >> > >> >> number of steps. Note that you need to add your
>> code
>> > > > > signing key to
>> > > > > > > > >> > >> >> the KEYS file in SVN (that's not very hard to
>> do). I
>> > > > > think it's fine
>> > > > > > > > >> > >> >> to hand off the process to others after the VOTE
>> but it
>> > > > > would be
>> > > > > > > > >> > >> >> tricky to have multiple RMs involved with
>> producing the
>> > > > > source and
>> > > > > > > > >> > >> >> binary artifacts for the vote
>> > > > > > > > >> > >> >>
>> > > > > > > > >> > >> >> On Tue, Sep 17, 2019 at 10:55 PM Micah Kornfield
>> <
>> > > > > > > > >> > >> emkornfi...@gmail.com> wrote:
>> > > > > > > > >> > >> >> >
>> > > > > > > > >> > >> >> > SGTM, as well.
>> > > > > > > > >> > >> >> >
>> > > > > > > > >> > >> >> > I should have a little bit of time next week
>> if I can
>> > > > > help as RM but
>> > > > > > > > >> > >> I have
>> > > > > > > > >> > >> >> > a couple of concerns:
>> > > > > > > > >> > >> >> > 1.  In the past I've had trouble downloading
>> and
>> > > > > validating
>> > > > > > > > >> > >> releases. I'm a
>> > > > > > > > >> > >> >> > bit worried, that I might have similar
>> problems doing
>> > > > > the necessary
>> > > > > > > > >> > >> uploads.
>> > > > > > > > >> > >> >> > 2.  My internet connection will likely be not
>> great, I
>> > > > > don't know if
>> > > > > > > > >> > >&g

Re: Timeline for 0.15.0 release

2019-09-25 Thread Wes McKinney
>> > >> >>
> > > > > > > >> > >> >> The process should be well documented at this point but
> > > > there are a
> > > > > > > >> > >> >> number of steps. Note that you need to add your code
> > > > signing key to
> > > > > > > >> > >> >> the KEYS file in SVN (that's not very hard to do). I
> > > > think it's fine
> > > > > > > >> > >> >> to hand off the process to others after the VOTE but it
> > > > would be
> > > > > > > >> > >> >> tricky to have multiple RMs involved with producing the
> > > > source and
> > > > > > > >> > >> >> binary artifacts for the vote
> > > > > > > >> > >> >>
> > > > > > > >> > >> >> On Tue, Sep 17, 2019 at 10:55 PM Micah Kornfield <
> > > > > > > >> > >> emkornfi...@gmail.com> wrote:
> > > > > > > >> > >> >> >
> > > > > > > >> > >> >> > SGTM, as well.
> > > > > > > >> > >> >> >
> > > > > > > >> > >> >> > I should have a little bit of time next week if I can
> > > > help as RM but
> > > > > > > >> > >> I have
> > > > > > > >> > >> >> > a couple of concerns:
> > > > > > > >> > >> >> > 1.  In the past I've had trouble downloading and
> > > > validating
> > > > > > > >> > >> releases. I'm a
> > > > > > > >> > >> >> > bit worried, that I might have similar problems doing
> > > > the necessary
> > > > > > > >> > >> uploads.
> > > > > > > >> > >> >> > 2.  My internet connection will likely be not great, 
> > > > > > > >> > >> >> > I
> > > > don't know if
> > > > > > > >> > >> this
> > > > > > > >> > >> >> > would make it even less likely to be successful.
> > > > > > > >> > >> >> >
> > > > > > > >> > >> >> > Does it become problematic if somehow I would have to
> > > > abandon the
> > > > > > > >> > >> process
> > > > > > > >> > >> >> > mid-release?  Is there anyone who could serve as a
> > > > backup?  Are the
> > > > > > > >> > >> steps
> > > > > > > >> > >> >> > well documented?
> > > > > > > >> > >> >> >
> > > > > > > >> > >> >> > Thanks,
> > > > > > > >> > >> >> > Micah
> > > > > > > >> > >> >> >
> > > > > > > >> > >> >> > On Tue, Sep 17, 2019 at 4:25 PM Neal Richardson <
> > > > > > > >> > >> neal.p.richard...@gmail.com>
> > > > > > > >> > >> >> > wrote:
> > > > > > > >> > >> >> >
> > > > > > > >> > >> >> > > Sounds good to me.
> > > > > > > >> > >> >> > >
> > > > > > > >> > >> >> > > Do we have a release manager yet? Any volunteers?
> > > > > > > >> > >> >> > >
> > > > > > > >> > >> >> > > Neal
> > > > > > > >> > >> >> > >
> > > > > > > >> > >> >> > > On Tue, Sep 17, 2019 at 4:06 PM Wes McKinney <
> > > > wesmck...@gmail.com>
> > > > > > > >> > >> wrote:
> > > > > > > >> > >> >> > >
> > > > > > > >> > >> >> > > > hi all,
> > > > > > > >> > >> >> > > >
> > > > > > > >> > >> >> > > > It looks like we're drawing close to be able to

Re: Timeline for 0.15.0 release

2019-09-25 Thread Micah Kornfield
guring
> > > > all
> > > > > > > > >> > > the CI backends can be time consuming.
> > > > > > > > >> > >
> > > > > > > > >> > >>
> > > > > > > > >> > >> > Thanks,
> > > > > > > > >> > >> > Micah
> > > > > > > > >> > >> >
> > > > > > > > >> > >> > [1]
> > > > > > > > >> > >>
> > > > >
> > > >
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
> > > > > > > > >> > >> >
> > > > > > > > >> > >> > On Wed, Sep 18, 2019 at 8:29 AM Wes McKinney <
> > > > > wesmck...@gmail.com>
> > > > > > > > >> > >> wrote:
> > > > > > > > >> > >> >>
> > > > > > > > >> > >> >> The process should be well documented at this
> point but
> > > > > there are a
> > > > > > > > >> > >> >> number of steps. Note that you need to add your
> code
> > > > > signing key to
> > > > > > > > >> > >> >> the KEYS file in SVN (that's not very hard to
> do). I
> > > > > think it's fine
> > > > > > > > >> > >> >> to hand off the process to others after the VOTE
> but it
> > > > > would be
> > > > > > > > >> > >> >> tricky to have multiple RMs involved with
> producing the
> > > > > source and
> > > > > > > > >> > >> >> binary artifacts for the vote
> > > > > > > > >> > >> >>
> > > > > > > > >> > >> >> On Tue, Sep 17, 2019 at 10:55 PM Micah Kornfield <
> > > > > > > > >> > >> emkornfi...@gmail.com> wrote:
> > > > > > > > >> > >> >> >
> > > > > > > > >> > >> >> > SGTM, as well.
> > > > > > > > >> > >> >> >
> > > > > > > > >> > >> >> > I should have a little bit of time next week if
> I can
> > > > > help as RM but
> > > > > > > > >> > >> I have
> > > > > > > > >> > >> >> > a couple of concerns:
> > > > > > > > >> > >> >> > 1.  In the past I've had trouble downloading and
> > > > > validating
> > > > > > > > >> > >> releases. I'm a
> > > > > > > > >> > >> >> > bit worried, that I might have similar problems
> doing
> > > > > the necessary
> > > > > > > > >> > >> uploads.
> > > > > > > > >> > >> >> > 2.  My internet connection will likely be not
> great, I
> > > > > don't know if
> > > > > > > > >> > >> this
> > > > > > > > >> > >> >> > would make it even less likely to be successful.
> > > > > > > > >> > >> >> >
> > > > > > > > >> > >> >> > Does it become problematic if somehow I would
> have to
> > > > > abandon the
> > > > > > > > >> > >> process
> > > > > > > > >> > >> >> > mid-release?  Is there anyone who could serve
> as a
> > > > > backup?  Are the
> > > > > > > > >> > >> steps
> > > > > > > > >> > >> >> > well documented?
> > > > > > > > >> > >> >> >
> > > > > > > > >> > >> >> > Thanks,
> > > > > > > > >> > >> >> > Micah
> > > > > > > > >> > >> >> >
> > > > > > > > >> > >> >> > On Tue, Sep 17, 2019 at 4:25 PM Neal Richardson
> <
> > > > > > > > >> > >> nea

Re: Timeline for 0.15.0 release

2019-09-25 Thread Neal Richardson
e:
> > > > > > >> > >> >> >
> > > > > > >> > >> >> > SGTM, as well.
> > > > > > >> > >> >> >
> > > > > > >> > >> >> > I should have a little bit of time next week if I can
> > > help as RM but
> > > > > > >> > >> I have
> > > > > > >> > >> >> > a couple of concerns:
> > > > > > >> > >> >> > 1.  In the past I've had trouble downloading and
> > > validating
> > > > > > >> > >> releases. I'm a
> > > > > > >> > >> >> > bit worried, that I might have similar problems doing
> > > the necessary
> > > > > > >> > >> uploads.
> > > > > > >> > >> >> > 2.  My internet connection will likely be not great, I
> > > don't know if
> > > > > > >> > >> this
> > > > > > >> > >> >> > would make it even less likely to be successful.
> > > > > > >> > >> >> >
> > > > > > >> > >> >> > Does it become problematic if somehow I would have to
> > > abandon the
> > > > > > >> > >> process
> > > > > > >> > >> >> > mid-release?  Is there anyone who could serve as a
> > > backup?  Are the
> > > > > > >> > >> steps
> > > > > > >> > >> >> > well documented?
> > > > > > >> > >> >> >
> > > > > > >> > >> >> > Thanks,
> > > > > > >> > >> >> > Micah
> > > > > > >> > >> >> >
> > > > > > >> > >> >> > On Tue, Sep 17, 2019 at 4:25 PM Neal Richardson <
> > > > > > >> > >> neal.p.richard...@gmail.com>
> > > > > > >> > >> >> > wrote:
> > > > > > >> > >> >> >
> > > > > > >> > >> >> > > Sounds good to me.
> > > > > > >> > >> >> > >
> > > > > > >> > >> >> > > Do we have a release manager yet? Any volunteers?
> > > > > > >> > >> >> > >
> > > > > > >> > >> >> > > Neal
> > > > > > >> > >> >> > >
> > > > > > >> > >> >> > > On Tue, Sep 17, 2019 at 4:06 PM Wes McKinney <
> > > wesmck...@gmail.com>
> > > > > > >> > >> wrote:
> > > > > > >> > >> >> > >
> > > > > > >> > >> >> > > > hi all,
> > > > > > >> > >> >> > > >
> > > > > > >> > >> >> > > > It looks like we're drawing close to be able to
> > > make the 0.15.0
> > > > > > >> > >> >> > > > release. I would suggest "pencils down" at the end
> > > of this week
> > > > > > >> > >> and
> > > > > > >> > >> >> > > > see if a release candidate can be produced next
> > > Monday September
> > > > > > >> > >> 23.
> > > > > > >> > >> >> > > > Any thoughts or objections?
> > > > > > >> > >> >> > > >
> > > > > > >> > >> >> > > > Thanks,
> > > > > > >> > >> >> > > > Wes
> > > > > > >> > >> >> > > >
> > > > > > >> > >> >> > > > On Wed, Sep 11, 2019 at 11:23 AM Wes McKinney <
> > > > > > >> > >> wesmck...@gmail.com>
> > > > > > >> > >> >> > > wrote:
> > > > > > >> > >> >> > > > >
> > > > > > >> > >> >> > > > > hi Eric -- yes, that's correct. I'm planning to
> &g

Re: Timeline for 0.15.0 release

2019-09-24 Thread Andy Grove
t; > > On Thu, Sep 19, 2019 at 5:52 PM Wes McKinney <
> > wesmck...@gmail.com> wrote:
> > > > > >> > >
> > > > > >> > >> On Thu, Sep 19, 2019 at 12:13 AM Micah Kornfield <
> > emkornfi...@gmail.com>
> > > > > >> > >> wrote:
> > > > > >> > >> >>
> > > > > >> > >> >> The process should be well documented at this point but
> > there are a
> > > > > >> > >> >> number of steps.
> > > > > >> > >> >
> > > > > >> > >> > Is [1] the up-to-date documentation for the release?
>  Are
> > there
> > > > > >> > >> instructions for the adding the code signing Key to SVN?
> > > > > >> > >> >
> > > > > >> > >> > I will make a go of it.  i will try to mitigate any
> > internet issues by
> > > > > >> > >> doing the process for a cloud instance (I assume that isn't
> > a problem?).
> > > > > >> > >> >
> > > > > >> > >>
> > > > > >> > >> Setting up a new cloud environment suitable for producing
> an
> > RC may be
> > > > > >> > >> time consuming, but you are welcome to try. Krisztian --
> are
> > you
> > > > > >> > >> available next week to help Micah and potentially take over
> > producing
> > > > > >> > >> the RC if there are issues?
> > > > > >> > >>
> > > > > >> > > Sure, I'll be available next week. We can also grant access
> to
> > > > > >> > > https://github.com/ursa-labs/crossbow because configuring
> all
> > > > > >> > > the CI backends can be time consuming.
> > > > > >> > >
> > > > > >> > >>
> > > > > >> > >> > Thanks,
> > > > > >> > >> > Micah
> > > > > >> > >> >
> > > > > >> > >> > [1]
> > > > > >> > >>
> >
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
> > > > > >> > >> >
> > > > > >> > >> > On Wed, Sep 18, 2019 at 8:29 AM Wes McKinney <
> > wesmck...@gmail.com>
> > > > > >> > >> wrote:
> > > > > >> > >> >>
> > > > > >> > >> >> The process should be well documented at this point but
> > there are a
> > > > > >> > >> >> number of steps. Note that you need to add your code
> > signing key to
> > > > > >> > >> >> the KEYS file in SVN (that's not very hard to do). I
> > think it's fine
> > > > > >> > >> >> to hand off the process to others after the VOTE but it
> > would be
> > > > > >> > >> >> tricky to have multiple RMs involved with producing the
> > source and
> > > > > >> > >> >> binary artifacts for the vote
> > > > > >> > >> >>
> > > > > >> > >> >> On Tue, Sep 17, 2019 at 10:55 PM Micah Kornfield <
> > > > > >> > >> emkornfi...@gmail.com> wrote:
> > > > > >> > >> >> >
> > > > > >> > >> >> > SGTM, as well.
> > > > > >> > >> >> >
> > > > > >> > >> >> > I should have a little bit of time next week if I can
> > help as RM but
> > > > > >> > >> I have
> > > > > >> > >> >> > a couple of concerns:
> > > > > >> > >> >> > 1.  In the past I've had trouble downloading and
> > validating
> > > > > >> > >> releases. I'm a
> > > > > >> > >> >> > bit worried, that I might have similar problems doing
> > the necessary
> > > > > >> > >> uploads.
> > > > > >> > >> >> > 2.  My internet connection will likely be not great, I
> > don't know if
> > > > > >> > >> this
> >

Re: Timeline for 0.15.0 release

2019-09-24 Thread Micah Kornfield
; instructions for the adding the code signing Key to SVN?
> > > > >> > >> >
> > > > >> > >> > I will make a go of it.  i will try to mitigate any
> internet issues by
> > > > >> > >> doing the process for a cloud instance (I assume that isn't
> a problem?).
> > > > >> > >> >
> > > > >> > >>
> > > > >> > >> Setting up a new cloud environment suitable for producing an
> RC may be
> > > > >> > >> time consuming, but you are welcome to try. Krisztian -- are
> you
> > > > >> > >> available next week to help Micah and potentially take over
> producing
> > > > >> > >> the RC if there are issues?
> > > > >> > >>
> > > > >> > > Sure, I'll be available next week. We can also grant access to
> > > > >> > > https://github.com/ursa-labs/crossbow because configuring all
> > > > >> > > the CI backends can be time consuming.
> > > > >> > >
> > > > >> > >>
> > > > >> > >> > Thanks,
> > > > >> > >> > Micah
> > > > >> > >> >
> > > > >> > >> > [1]
> > > > >> > >>
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
> > > > >> > >> >
> > > > >> > >> > On Wed, Sep 18, 2019 at 8:29 AM Wes McKinney <
> wesmck...@gmail.com>
> > > > >> > >> wrote:
> > > > >> > >> >>
> > > > >> > >> >> The process should be well documented at this point but
> there are a
> > > > >> > >> >> number of steps. Note that you need to add your code
> signing key to
> > > > >> > >> >> the KEYS file in SVN (that's not very hard to do). I
> think it's fine
> > > > >> > >> >> to hand off the process to others after the VOTE but it
> would be
> > > > >> > >> >> tricky to have multiple RMs involved with producing the
> source and
> > > > >> > >> >> binary artifacts for the vote
> > > > >> > >> >>
> > > > >> > >> >> On Tue, Sep 17, 2019 at 10:55 PM Micah Kornfield <
> > > > >> > >> emkornfi...@gmail.com> wrote:
> > > > >> > >> >> >
> > > > >> > >> >> > SGTM, as well.
> > > > >> > >> >> >
> > > > >> > >> >> > I should have a little bit of time next week if I can
> help as RM but
> > > > >> > >> I have
> > > > >> > >> >> > a couple of concerns:
> > > > >> > >> >> > 1.  In the past I've had trouble downloading and
> validating
> > > > >> > >> releases. I'm a
> > > > >> > >> >> > bit worried, that I might have similar problems doing
> the necessary
> > > > >> > >> uploads.
> > > > >> > >> >> > 2.  My internet connection will likely be not great, I
> don't know if
> > > > >> > >> this
> > > > >> > >> >> > would make it even less likely to be successful.
> > > > >> > >> >> >
> > > > >> > >> >> > Does it become problematic if somehow I would have to
> abandon the
> > > > >> > >> process
> > > > >> > >> >> > mid-release?  Is there anyone who could serve as a
> backup?  Are the
> > > > >> > >> steps
> > > > >> > >> >> > well documented?
> > > > >> > >> >> >
> > > > >> > >> >> > Thanks,
> > > > >> > >> >> > Micah
> > > > >> > >> >> >
> > > > >> > >> >> > On Tue, Sep 17, 2019 at 4:25 PM Neal Richardson <
> > > > >> > >> neal.p.richard...@gmail.com>
> > > > >> > >> >> > wrote:
> > > > >> > >> >> >
> > > > >> > >> >> &g

Re: Timeline for 0.15.0 release

2019-09-24 Thread Wes McKinney
;> > >>
> > > >> > >> Setting up a new cloud environment suitable for producing an RC 
> > > >> > >> may be
> > > >> > >> time consuming, but you are welcome to try. Krisztian -- are you
> > > >> > >> available next week to help Micah and potentially take over 
> > > >> > >> producing
> > > >> > >> the RC if there are issues?
> > > >> > >>
> > > >> > > Sure, I'll be available next week. We can also grant access to
> > > >> > > https://github.com/ursa-labs/crossbow because configuring all
> > > >> > > the CI backends can be time consuming.
> > > >> > >
> > > >> > >>
> > > >> > >> > Thanks,
> > > >> > >> > Micah
> > > >> > >> >
> > > >> > >> > [1]
> > > >> > >> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
> > > >> > >> >
> > > >> > >> > On Wed, Sep 18, 2019 at 8:29 AM Wes McKinney 
> > > >> > >> > 
> > > >> > >> wrote:
> > > >> > >> >>
> > > >> > >> >> The process should be well documented at this point but there 
> > > >> > >> >> are a
> > > >> > >> >> number of steps. Note that you need to add your code signing 
> > > >> > >> >> key to
> > > >> > >> >> the KEYS file in SVN (that's not very hard to do). I think 
> > > >> > >> >> it's fine
> > > >> > >> >> to hand off the process to others after the VOTE but it would 
> > > >> > >> >> be
> > > >> > >> >> tricky to have multiple RMs involved with producing the source 
> > > >> > >> >> and
> > > >> > >> >> binary artifacts for the vote
> > > >> > >> >>
> > > >> > >> >> On Tue, Sep 17, 2019 at 10:55 PM Micah Kornfield <
> > > >> > >> emkornfi...@gmail.com> wrote:
> > > >> > >> >> >
> > > >> > >> >> > SGTM, as well.
> > > >> > >> >> >
> > > >> > >> >> > I should have a little bit of time next week if I can help 
> > > >> > >> >> > as RM but
> > > >> > >> I have
> > > >> > >> >> > a couple of concerns:
> > > >> > >> >> > 1.  In the past I've had trouble downloading and validating
> > > >> > >> releases. I'm a
> > > >> > >> >> > bit worried, that I might have similar problems doing the 
> > > >> > >> >> > necessary
> > > >> > >> uploads.
> > > >> > >> >> > 2.  My internet connection will likely be not great, I don't 
> > > >> > >> >> > know if
> > > >> > >> this
> > > >> > >> >> > would make it even less likely to be successful.
> > > >> > >> >> >
> > > >> > >> >> > Does it become problematic if somehow I would have to 
> > > >> > >> >> > abandon the
> > > >> > >> process
> > > >> > >> >> > mid-release?  Is there anyone who could serve as a backup?  
> > > >> > >> >> > Are the
> > > >> > >> steps
> > > >> > >> >> > well documented?
> > > >> > >> >> >
> > > >> > >> >> > Thanks,
> > > >> > >> >> > Micah
> > > >> > >> >> >
> > > >> > >> >> > On Tue, Sep 17, 2019 at 4:25 PM Neal Richardson <
> > > >> > >> neal.p.richard...@gmail.com>
> > > >> > >> >> > wrote:
> > > >> > >> >> >
> > > >> > >> >> > > Sounds good to me.
> > > >> > >> >> > >
> > > >> > >> >> 

Re: Timeline for 0.15.0 release

2019-09-24 Thread Wes McKinney
>
> > >> > >>
> > >> > >> > Thanks,
> > >> > >> > Micah
> > >> > >> >
> > >> > >> > [1]
> > >> > >> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
> > >> > >> >
> > >> > >> > On Wed, Sep 18, 2019 at 8:29 AM Wes McKinney 
> > >> > >> wrote:
> > >> > >> >>
> > >> > >> >> The process should be well documented at this point but there 
> > >> > >> >> are a
> > >> > >> >> number of steps. Note that you need to add your code signing key 
> > >> > >> >> to
> > >> > >> >> the KEYS file in SVN (that's not very hard to do). I think it's 
> > >> > >> >> fine
> > >> > >> >> to hand off the process to others after the VOTE but it would be
> > >> > >> >> tricky to have multiple RMs involved with producing the source 
> > >> > >> >> and
> > >> > >> >> binary artifacts for the vote
> > >> > >> >>
> > >> > >> >> On Tue, Sep 17, 2019 at 10:55 PM Micah Kornfield <
> > >> > >> emkornfi...@gmail.com> wrote:
> > >> > >> >> >
> > >> > >> >> > SGTM, as well.
> > >> > >> >> >
> > >> > >> >> > I should have a little bit of time next week if I can help as 
> > >> > >> >> > RM but
> > >> > >> I have
> > >> > >> >> > a couple of concerns:
> > >> > >> >> > 1.  In the past I've had trouble downloading and validating
> > >> > >> releases. I'm a
> > >> > >> >> > bit worried, that I might have similar problems doing the 
> > >> > >> >> > necessary
> > >> > >> uploads.
> > >> > >> >> > 2.  My internet connection will likely be not great, I don't 
> > >> > >> >> > know if
> > >> > >> this
> > >> > >> >> > would make it even less likely to be successful.
> > >> > >> >> >
> > >> > >> >> > Does it become problematic if somehow I would have to abandon 
> > >> > >> >> > the
> > >> > >> process
> > >> > >> >> > mid-release?  Is there anyone who could serve as a backup?  
> > >> > >> >> > Are the
> > >> > >> steps
> > >> > >> >> > well documented?
> > >> > >> >> >
> > >> > >> >> > Thanks,
> > >> > >> >> > Micah
> > >> > >> >> >
> > >> > >> >> > On Tue, Sep 17, 2019 at 4:25 PM Neal Richardson <
> > >> > >> neal.p.richard...@gmail.com>
> > >> > >> >> > wrote:
> > >> > >> >> >
> > >> > >> >> > > Sounds good to me.
> > >> > >> >> > >
> > >> > >> >> > > Do we have a release manager yet? Any volunteers?
> > >> > >> >> > >
> > >> > >> >> > > Neal
> > >> > >> >> > >
> > >> > >> >> > > On Tue, Sep 17, 2019 at 4:06 PM Wes McKinney 
> > >> > >> >> > > 
> > >> > >> wrote:
> > >> > >> >> > >
> > >> > >> >> > > > hi all,
> > >> > >> >> > > >
> > >> > >> >> > > > It looks like we're drawing close to be able to make the 
> > >> > >> >> > > > 0.15.0
> > >> > >> >> > > > release. I would suggest "pencils down" at the end of this 
> > >> > >> >> > > > week
> > >> > >> and
> > >> > >> >> > > > see if a release candidate can be produced next Monday 
> > >> > >> >> > > > September
> > >> > >> 23.
> > >> > >> >> > &

Re: Timeline for 0.15.0 release

2019-09-24 Thread Wes McKinney
it's 
> >> > >> >> fine
> >> > >> >> to hand off the process to others after the VOTE but it would be
> >> > >> >> tricky to have multiple RMs involved with producing the source and
> >> > >> >> binary artifacts for the vote
> >> > >> >>
> >> > >> >> On Tue, Sep 17, 2019 at 10:55 PM Micah Kornfield <
> >> > >> emkornfi...@gmail.com> wrote:
> >> > >> >> >
> >> > >> >> > SGTM, as well.
> >> > >> >> >
> >> > >> >> > I should have a little bit of time next week if I can help as RM 
> >> > >> >> > but
> >> > >> I have
> >> > >> >> > a couple of concerns:
> >> > >> >> > 1.  In the past I've had trouble downloading and validating
> >> > >> releases. I'm a
> >> > >> >> > bit worried, that I might have similar problems doing the 
> >> > >> >> > necessary
> >> > >> uploads.
> >> > >> >> > 2.  My internet connection will likely be not great, I don't 
> >> > >> >> > know if
> >> > >> this
> >> > >> >> > would make it even less likely to be successful.
> >> > >> >> >
> >> > >> >> > Does it become problematic if somehow I would have to abandon the
> >> > >> process
> >> > >> >> > mid-release?  Is there anyone who could serve as a backup?  Are 
> >> > >> >> > the
> >> > >> steps
> >> > >> >> > well documented?
> >> > >> >> >
> >> > >> >> > Thanks,
> >> > >> >> > Micah
> >> > >> >> >
> >> > >> >> > On Tue, Sep 17, 2019 at 4:25 PM Neal Richardson <
> >> > >> neal.p.richard...@gmail.com>
> >> > >> >> > wrote:
> >> > >> >> >
> >> > >> >> > > Sounds good to me.
> >> > >> >> > >
> >> > >> >> > > Do we have a release manager yet? Any volunteers?
> >> > >> >> > >
> >> > >> >> > > Neal
> >> > >> >> > >
> >> > >> >> > > On Tue, Sep 17, 2019 at 4:06 PM Wes McKinney 
> >> > >> >> > > 
> >> > >> wrote:
> >> > >> >> > >
> >> > >> >> > > > hi all,
> >> > >> >> > > >
> >> > >> >> > > > It looks like we're drawing close to be able to make the 
> >> > >> >> > > > 0.15.0
> >> > >> >> > > > release. I would suggest "pencils down" at the end of this 
> >> > >> >> > > > week
> >> > >> and
> >> > >> >> > > > see if a release candidate can be produced next Monday 
> >> > >> >> > > > September
> >> > >> 23.
> >> > >> >> > > > Any thoughts or objections?
> >> > >> >> > > >
> >> > >> >> > > > Thanks,
> >> > >> >> > > > Wes
> >> > >> >> > > >
> >> > >> >> > > > On Wed, Sep 11, 2019 at 11:23 AM Wes McKinney <
> >> > >> wesmck...@gmail.com>
> >> > >> >> > > wrote:
> >> > >> >> > > > >
> >> > >> >> > > > > hi Eric -- yes, that's correct. I'm planning to amend the
> >> > >> Format docs
> >> > >> >> > > > > today regarding the EOS issue and also update the C++ 
> >> > >> >> > > > > library
> >> > >> >> > > > >
> >> > >> >> > > > > On Wed, Sep 11, 2019 at 11:21 AM Eric Erhardt
> >> > >> >> > > > >  wrote:
> >> > >> >> > > > > >
> >> > >> >> > > > > > I assume the plan is to merge the
> >> > >> ARROW-6313-flatbuffer-alignment
> >> &

Re: Timeline for 0.15.0 release

2019-09-24 Thread Wes McKinney
gt;> > >> >> > 1.  In the past I've had trouble downloading and validating
>> > >> releases. I'm a
>> > >> >> > bit worried, that I might have similar problems doing the necessary
>> > >> uploads.
>> > >> >> > 2.  My internet connection will likely be not great, I don't know 
>> > >> >> > if
>> > >> this
>> > >> >> > would make it even less likely to be successful.
>> > >> >> >
>> > >> >> > Does it become problematic if somehow I would have to abandon the
>> > >> process
>> > >> >> > mid-release?  Is there anyone who could serve as a backup?  Are the
>> > >> steps
>> > >> >> > well documented?
>> > >> >> >
>> > >> >> > Thanks,
>> > >> >> > Micah
>> > >> >> >
>> > >> >> > On Tue, Sep 17, 2019 at 4:25 PM Neal Richardson <
>> > >> neal.p.richard...@gmail.com>
>> > >> >> > wrote:
>> > >> >> >
>> > >> >> > > Sounds good to me.
>> > >> >> > >
>> > >> >> > > Do we have a release manager yet? Any volunteers?
>> > >> >> > >
>> > >> >> > > Neal
>> > >> >> > >
>> > >> >> > > On Tue, Sep 17, 2019 at 4:06 PM Wes McKinney 
>> > >> >> > > 
>> > >> wrote:
>> > >> >> > >
>> > >> >> > > > hi all,
>> > >> >> > > >
>> > >> >> > > > It looks like we're drawing close to be able to make the 0.15.0
>> > >> >> > > > release. I would suggest "pencils down" at the end of this week
>> > >> and
>> > >> >> > > > see if a release candidate can be produced next Monday 
>> > >> >> > > > September
>> > >> 23.
>> > >> >> > > > Any thoughts or objections?
>> > >> >> > > >
>> > >> >> > > > Thanks,
>> > >> >> > > > Wes
>> > >> >> > > >
>> > >> >> > > > On Wed, Sep 11, 2019 at 11:23 AM Wes McKinney <
>> > >> wesmck...@gmail.com>
>> > >> >> > > wrote:
>> > >> >> > > > >
>> > >> >> > > > > hi Eric -- yes, that's correct. I'm planning to amend the
>> > >> Format docs
>> > >> >> > > > > today regarding the EOS issue and also update the C++ library
>> > >> >> > > > >
>> > >> >> > > > > On Wed, Sep 11, 2019 at 11:21 AM Eric Erhardt
>> > >> >> > > > >  wrote:
>> > >> >> > > > > >
>> > >> >> > > > > > I assume the plan is to merge the
>> > >> ARROW-6313-flatbuffer-alignment
>> > >> >> > > > branch into master before the 0.15 release, correct?
>> > >> >> > > > > >
>> > >> >> > > > > > BTW - I believe the C# alignment changes are ready to be
>> > >> merged into
>> > >> >> > > > the alignment branch -
>> > >> https://github.com/apache/arrow/pull/5280/
>> > >> >> > > > > >
>> > >> >> > > > > > Eric
>> > >> >> > > > > >
>> > >> >> > > > > > -Original Message-
>> > >> >> > > > > > From: Micah Kornfield 
>> > >> >> > > > > > Sent: Tuesday, September 10, 2019 10:24 PM
>> > >> >> > > > > > To: Wes McKinney 
>> > >> >> > > > > > Cc: dev ; niki.lj 
>> > >> >> > > > > > 
>> > >> >> > > > > > Subject: Re: Timeline for 0.15.0 release
>> > >> >> > > > > >
>> > >> >> > > > > > I should have a little more bandwidth to help with some of
>> > >> the
>> > >> >> > > > pack

Re: Timeline for 0.15.0 release

2019-09-22 Thread Micah Kornfield
> Does it become problematic if somehow I would have to abandon the
> > >> process
> > >> >> > mid-release?  Is there anyone who could serve as a backup?  Are
> the
> > >> steps
> > >> >> > well documented?
> > >> >> >
> > >> >> > Thanks,
> > >> >> > Micah
> > >> >> >
> > >> >> > On Tue, Sep 17, 2019 at 4:25 PM Neal Richardson <
> > >> neal.p.richard...@gmail.com>
> > >> >> > wrote:
> > >> >> >
> > >> >> > > Sounds good to me.
> > >> >> > >
> > >> >> > > Do we have a release manager yet? Any volunteers?
> > >> >> > >
> > >> >> > > Neal
> > >> >> > >
> > >> >> > > On Tue, Sep 17, 2019 at 4:06 PM Wes McKinney <
> wesmck...@gmail.com>
> > >> wrote:
> > >> >> > >
> > >> >> > > > hi all,
> > >> >> > > >
> > >> >> > > > It looks like we're drawing close to be able to make the
> 0.15.0
> > >> >> > > > release. I would suggest "pencils down" at the end of this
> week
> > >> and
> > >> >> > > > see if a release candidate can be produced next Monday
> September
> > >> 23.
> > >> >> > > > Any thoughts or objections?
> > >> >> > > >
> > >> >> > > > Thanks,
> > >> >> > > > Wes
> > >> >> > > >
> > >> >> > > > On Wed, Sep 11, 2019 at 11:23 AM Wes McKinney <
> > >> wesmck...@gmail.com>
> > >> >> > > wrote:
> > >> >> > > > >
> > >> >> > > > > hi Eric -- yes, that's correct. I'm planning to amend the
> > >> Format docs
> > >> >> > > > > today regarding the EOS issue and also update the C++
> library
> > >> >> > > > >
> > >> >> > > > > On Wed, Sep 11, 2019 at 11:21 AM Eric Erhardt
> > >> >> > > > >  wrote:
> > >> >> > > > > >
> > >> >> > > > > > I assume the plan is to merge the
> > >> ARROW-6313-flatbuffer-alignment
> > >> >> > > > branch into master before the 0.15 release, correct?
> > >> >> > > > > >
> > >> >> > > > > > BTW - I believe the C# alignment changes are ready to be
> > >> merged into
> > >> >> > > > the alignment branch -
> > >> https://github.com/apache/arrow/pull/5280/
> > >> >> > > > > >
> > >> >> > > > > > Eric
> > >> >> > > > > >
> > >> >> > > > > > -Original Message-
> > >> >> > > > > > From: Micah Kornfield 
> > >> >> > > > > > Sent: Tuesday, September 10, 2019 10:24 PM
> > >> >> > > > > > To: Wes McKinney 
> > >> >> > > > > > Cc: dev ; niki.lj <
> niki...@aliyun.com>
> > >> >> > > > > > Subject: Re: Timeline for 0.15.0 release
> > >> >> > > > > >
> > >> >> > > > > > I should have a little more bandwidth to help with some
> of
> > >> the
> > >> >> > > > packaging starting tomorrow and going into the weekend.
> > >> >> > > > > >
> > >> >> > > > > > On Tuesday, September 10, 2019, Wes McKinney <
> > >> wesmck...@gmail.com>
> > >> >> > > > wrote:
> > >> >> > > > > >
> > >> >> > > > > > > Hi folks,
> > >> >> > > > > > >
> > >> >> > > > > > > With the state of nightly packaging and integration
> builds
> > >> things
> > >> >> > > > > > > aren't looking too good for being in release readiness
> by
> > >> the end
> > >> >> > > of
> > >> >> > > > > > > this week but maybe I'm wrong. I'm planning to be
> working
> > &

Re: Timeline for 0.15.0 release

2019-09-22 Thread Andy Grove
gt; > >> >> > would make it even less likely to be successful.
> > >> >> >
> > >> >> > Does it become problematic if somehow I would have to abandon the
> > >> process
> > >> >> > mid-release?  Is there anyone who could serve as a backup?  Are
> the
> > >> steps
> > >> >> > well documented?
> > >> >> >
> > >> >> > Thanks,
> > >> >> > Micah
> > >> >> >
> > >> >> > On Tue, Sep 17, 2019 at 4:25 PM Neal Richardson <
> > >> neal.p.richard...@gmail.com>
> > >> >> > wrote:
> > >> >> >
> > >> >> > > Sounds good to me.
> > >> >> > >
> > >> >> > > Do we have a release manager yet? Any volunteers?
> > >> >> > >
> > >> >> > > Neal
> > >> >> > >
> > >> >> > > On Tue, Sep 17, 2019 at 4:06 PM Wes McKinney <
> wesmck...@gmail.com>
> > >> wrote:
> > >> >> > >
> > >> >> > > > hi all,
> > >> >> > > >
> > >> >> > > > It looks like we're drawing close to be able to make the
> 0.15.0
> > >> >> > > > release. I would suggest "pencils down" at the end of this
> week
> > >> and
> > >> >> > > > see if a release candidate can be produced next Monday
> September
> > >> 23.
> > >> >> > > > Any thoughts or objections?
> > >> >> > > >
> > >> >> > > > Thanks,
> > >> >> > > > Wes
> > >> >> > > >
> > >> >> > > > On Wed, Sep 11, 2019 at 11:23 AM Wes McKinney <
> > >> wesmck...@gmail.com>
> > >> >> > > wrote:
> > >> >> > > > >
> > >> >> > > > > hi Eric -- yes, that's correct. I'm planning to amend the
> > >> Format docs
> > >> >> > > > > today regarding the EOS issue and also update the C++
> library
> > >> >> > > > >
> > >> >> > > > > On Wed, Sep 11, 2019 at 11:21 AM Eric Erhardt
> > >> >> > > > >  wrote:
> > >> >> > > > > >
> > >> >> > > > > > I assume the plan is to merge the
> > >> ARROW-6313-flatbuffer-alignment
> > >> >> > > > branch into master before the 0.15 release, correct?
> > >> >> > > > > >
> > >> >> > > > > > BTW - I believe the C# alignment changes are ready to be
> > >> merged into
> > >> >> > > > the alignment branch -
> > >> https://github.com/apache/arrow/pull/5280/
> > >> >> > > > > >
> > >> >> > > > > > Eric
> > >> >> > > > > >
> > >> >> > > > > > -Original Message-
> > >> >> > > > > > From: Micah Kornfield 
> > >> >> > > > > > Sent: Tuesday, September 10, 2019 10:24 PM
> > >> >> > > > > > To: Wes McKinney 
> > >> >> > > > > > Cc: dev ; niki.lj <
> niki...@aliyun.com>
> > >> >> > > > > > Subject: Re: Timeline for 0.15.0 release
> > >> >> > > > > >
> > >> >> > > > > > I should have a little more bandwidth to help with some
> of
> > >> the
> > >> >> > > > packaging starting tomorrow and going into the weekend.
> > >> >> > > > > >
> > >> >> > > > > > On Tuesday, September 10, 2019, Wes McKinney <
> > >> wesmck...@gmail.com>
> > >> >> > > > wrote:
> > >> >> > > > > >
> > >> >> > > > > > > Hi folks,
> > >> >> > > > > > >
> > >> >> > > > > > > With the state of nightly packaging and integration
> builds
> > >> things
> > >> >> > > > > > > aren't looking too good for being in release readiness
> by
> > >> the end
> > >

Re: Timeline for 0.15.0 release

2019-09-21 Thread Wes McKinney
t; > >
> >> >> > > > It looks like we're drawing close to be able to make the 0.15.0
> >> >> > > > release. I would suggest "pencils down" at the end of this week
> >> and
> >> >> > > > see if a release candidate can be produced next Monday September
> >> 23.
> >> >> > > > Any thoughts or objections?
> >> >> > > >
> >> >> > > > Thanks,
> >> >> > > > Wes
> >> >> > > >
> >> >> > > > On Wed, Sep 11, 2019 at 11:23 AM Wes McKinney <
> >> wesmck...@gmail.com>
> >> >> > > wrote:
> >> >> > > > >
> >> >> > > > > hi Eric -- yes, that's correct. I'm planning to amend the
> >> Format docs
> >> >> > > > > today regarding the EOS issue and also update the C++ library
> >> >> > > > >
> >> >> > > > > On Wed, Sep 11, 2019 at 11:21 AM Eric Erhardt
> >> >> > > > >  wrote:
> >> >> > > > > >
> >> >> > > > > > I assume the plan is to merge the
> >> ARROW-6313-flatbuffer-alignment
> >> >> > > > branch into master before the 0.15 release, correct?
> >> >> > > > > >
> >> >> > > > > > BTW - I believe the C# alignment changes are ready to be
> >> merged into
> >> >> > > > the alignment branch -
> >> https://github.com/apache/arrow/pull/5280/
> >> >> > > > > >
> >> >> > > > > > Eric
> >> >> > > > > >
> >> >> > > > > > -Original Message-
> >> >> > > > > > From: Micah Kornfield 
> >> >> > > > > > Sent: Tuesday, September 10, 2019 10:24 PM
> >> >> > > > > > To: Wes McKinney 
> >> >> > > > > > Cc: dev ; niki.lj 
> >> >> > > > > > Subject: Re: Timeline for 0.15.0 release
> >> >> > > > > >
> >> >> > > > > > I should have a little more bandwidth to help with some of
> >> the
> >> >> > > > packaging starting tomorrow and going into the weekend.
> >> >> > > > > >
> >> >> > > > > > On Tuesday, September 10, 2019, Wes McKinney <
> >> wesmck...@gmail.com>
> >> >> > > > wrote:
> >> >> > > > > >
> >> >> > > > > > > Hi folks,
> >> >> > > > > > >
> >> >> > > > > > > With the state of nightly packaging and integration builds
> >> things
> >> >> > > > > > > aren't looking too good for being in release readiness by
> >> the end
> >> >> > > of
> >> >> > > > > > > this week but maybe I'm wrong. I'm planning to be working
> >> to close
> >> >> > > as
> >> >> > > > > > > many issues as I can and also to help with the ongoing
> >> alignment
> >> >> > > > fixes.
> >> >> > > > > > >
> >> >> > > > > > > Wes
> >> >> > > > > > >
> >> >> > > > > > > On Thu, Sep 5, 2019, 11:07 PM Micah Kornfield <
> >> >> > > emkornfi...@gmail.com
> >> >> > > > >
> >> >> > > > > > > wrote:
> >> >> > > > > > >
> >> >> > > > > > >> Just for reference [1] has a dashboard of the current
> >> issues:
> >> >> > > > > > >>
> >> >> > > > > > >>
> >> >> > > >
> >> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwi
> >> >> > > > > > >> ki.apache.org
> >> >> > > > %2Fconfluence%2Fdisplay%2FARROW%2FArrow%2B0.15.0%2BRelea
> >> >> > > > > > >> sedata=02%7C01%7CEric.Erhardt%40microsoft.com
> >> >> > > > %7Ccbead81a42104034
> >> >> > > > > > >>
> >> >> > > >
> >> a4f308d736678a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7

Re: Timeline for 0.15.0 release

2019-09-20 Thread Micah Kornfield
Thanks Krisztián and Wes,
I've gone ahead and started registering myself on all the packaging sites.

Is there any review process when adding my GPG key to the SVN file? [1]
doesn't seem to mention explicitly.

Thanks,
Micah

[1] https://www.apache.org/dev/version-control.html#https-svn

On Fri, Sep 20, 2019 at 5:01 PM Krisztián Szűcs 
wrote:

> On Thu, Sep 19, 2019 at 5:52 PM Wes McKinney  wrote:
>
>> On Thu, Sep 19, 2019 at 12:13 AM Micah Kornfield 
>> wrote:
>> >>
>> >> The process should be well documented at this point but there are a
>> >> number of steps.
>> >
>> > Is [1] the up-to-date documentation for the release?   Are there
>> instructions for the adding the code signing Key to SVN?
>> >
>> > I will make a go of it.  i will try to mitigate any internet issues by
>> doing the process for a cloud instance (I assume that isn't a problem?).
>> >
>>
>> Setting up a new cloud environment suitable for producing an RC may be
>> time consuming, but you are welcome to try. Krisztian -- are you
>> available next week to help Micah and potentially take over producing
>> the RC if there are issues?
>>
> Sure, I'll be available next week. We can also grant access to
> https://github.com/ursa-labs/crossbow because configuring all
> the CI backends can be time consuming.
>
>>
>> > Thanks,
>> > Micah
>> >
>> > [1]
>> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
>> >
>> > On Wed, Sep 18, 2019 at 8:29 AM Wes McKinney 
>> wrote:
>> >>
>> >> The process should be well documented at this point but there are a
>> >> number of steps. Note that you need to add your code signing key to
>> >> the KEYS file in SVN (that's not very hard to do). I think it's fine
>> >> to hand off the process to others after the VOTE but it would be
>> >> tricky to have multiple RMs involved with producing the source and
>> >> binary artifacts for the vote
>> >>
>> >> On Tue, Sep 17, 2019 at 10:55 PM Micah Kornfield <
>> emkornfi...@gmail.com> wrote:
>> >> >
>> >> > SGTM, as well.
>> >> >
>> >> > I should have a little bit of time next week if I can help as RM but
>> I have
>> >> > a couple of concerns:
>> >> > 1.  In the past I've had trouble downloading and validating
>> releases. I'm a
>> >> > bit worried, that I might have similar problems doing the necessary
>> uploads.
>> >> > 2.  My internet connection will likely be not great, I don't know if
>> this
>> >> > would make it even less likely to be successful.
>> >> >
>> >> > Does it become problematic if somehow I would have to abandon the
>> process
>> >> > mid-release?  Is there anyone who could serve as a backup?  Are the
>> steps
>> >> > well documented?
>> >> >
>> >> > Thanks,
>> >> > Micah
>> >> >
>> >> > On Tue, Sep 17, 2019 at 4:25 PM Neal Richardson <
>> neal.p.richard...@gmail.com>
>> >> > wrote:
>> >> >
>> >> > > Sounds good to me.
>> >> > >
>> >> > > Do we have a release manager yet? Any volunteers?
>> >> > >
>> >> > > Neal
>> >> > >
>> >> > > On Tue, Sep 17, 2019 at 4:06 PM Wes McKinney 
>> wrote:
>> >> > >
>> >> > > > hi all,
>> >> > > >
>> >> > > > It looks like we're drawing close to be able to make the 0.15.0
>> >> > > > release. I would suggest "pencils down" at the end of this week
>> and
>> >> > > > see if a release candidate can be produced next Monday September
>> 23.
>> >> > > > Any thoughts or objections?
>> >> > > >
>> >> > > > Thanks,
>> >> > > > Wes
>> >> > > >
>> >> > > > On Wed, Sep 11, 2019 at 11:23 AM Wes McKinney <
>> wesmck...@gmail.com>
>> >> > > wrote:
>> >> > > > >
>> >> > > > > hi Eric -- yes, that's correct. I'm planning to amend the
>> Format docs
>> >> > > > > today regarding the EOS issue and also update the C++ library
>> >> > > > >
>> >> > > > > On W

Re: Timeline for 0.15.0 release

2019-09-20 Thread Krisztián Szűcs
On Thu, Sep 19, 2019 at 5:52 PM Wes McKinney  wrote:

> On Thu, Sep 19, 2019 at 12:13 AM Micah Kornfield 
> wrote:
> >>
> >> The process should be well documented at this point but there are a
> >> number of steps.
> >
> > Is [1] the up-to-date documentation for the release?   Are there
> instructions for the adding the code signing Key to SVN?
> >
> > I will make a go of it.  i will try to mitigate any internet issues by
> doing the process for a cloud instance (I assume that isn't a problem?).
> >
>
> Setting up a new cloud environment suitable for producing an RC may be
> time consuming, but you are welcome to try. Krisztian -- are you
> available next week to help Micah and potentially take over producing
> the RC if there are issues?
>
Sure, I'll be available next week. We can also grant access to
https://github.com/ursa-labs/crossbow because configuring all
the CI backends can be time consuming.

>
> > Thanks,
> > Micah
> >
> > [1]
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
> >
> > On Wed, Sep 18, 2019 at 8:29 AM Wes McKinney 
> wrote:
> >>
> >> The process should be well documented at this point but there are a
> >> number of steps. Note that you need to add your code signing key to
> >> the KEYS file in SVN (that's not very hard to do). I think it's fine
> >> to hand off the process to others after the VOTE but it would be
> >> tricky to have multiple RMs involved with producing the source and
> >> binary artifacts for the vote
> >>
> >> On Tue, Sep 17, 2019 at 10:55 PM Micah Kornfield 
> wrote:
> >> >
> >> > SGTM, as well.
> >> >
> >> > I should have a little bit of time next week if I can help as RM but
> I have
> >> > a couple of concerns:
> >> > 1.  In the past I've had trouble downloading and validating releases.
> I'm a
> >> > bit worried, that I might have similar problems doing the necessary
> uploads.
> >> > 2.  My internet connection will likely be not great, I don't know if
> this
> >> > would make it even less likely to be successful.
> >> >
> >> > Does it become problematic if somehow I would have to abandon the
> process
> >> > mid-release?  Is there anyone who could serve as a backup?  Are the
> steps
> >> > well documented?
> >> >
> >> > Thanks,
> >> > Micah
> >> >
> >> > On Tue, Sep 17, 2019 at 4:25 PM Neal Richardson <
> neal.p.richard...@gmail.com>
> >> > wrote:
> >> >
> >> > > Sounds good to me.
> >> > >
> >> > > Do we have a release manager yet? Any volunteers?
> >> > >
> >> > > Neal
> >> > >
> >> > > On Tue, Sep 17, 2019 at 4:06 PM Wes McKinney 
> wrote:
> >> > >
> >> > > > hi all,
> >> > > >
> >> > > > It looks like we're drawing close to be able to make the 0.15.0
> >> > > > release. I would suggest "pencils down" at the end of this week
> and
> >> > > > see if a release candidate can be produced next Monday September
> 23.
> >> > > > Any thoughts or objections?
> >> > > >
> >> > > > Thanks,
> >> > > > Wes
> >> > > >
> >> > > > On Wed, Sep 11, 2019 at 11:23 AM Wes McKinney <
> wesmck...@gmail.com>
> >> > > wrote:
> >> > > > >
> >> > > > > hi Eric -- yes, that's correct. I'm planning to amend the
> Format docs
> >> > > > > today regarding the EOS issue and also update the C++ library
> >> > > > >
> >> > > > > On Wed, Sep 11, 2019 at 11:21 AM Eric Erhardt
> >> > > > >  wrote:
> >> > > > > >
> >> > > > > > I assume the plan is to merge the
> ARROW-6313-flatbuffer-alignment
> >> > > > branch into master before the 0.15 release, correct?
> >> > > > > >
> >> > > > > > BTW - I believe the C# alignment changes are ready to be
> merged into
> >> > > > the alignment branch -
> https://github.com/apache/arrow/pull/5280/
> >> > > > > >
> >> > > > > > Eric
> >> > > > > >
> >> > > > > > -Original Message-
> >>

Re: Timeline for 0.15.0 release

2019-09-19 Thread Wes McKinney
On Thu, Sep 19, 2019 at 12:13 AM Micah Kornfield  wrote:
>>
>> The process should be well documented at this point but there are a
>> number of steps.
>
> Is [1] the up-to-date documentation for the release?   Are there instructions 
> for the adding the code signing Key to SVN?
>
> I will make a go of it.  i will try to mitigate any internet issues by doing 
> the process for a cloud instance (I assume that isn't a problem?).
>

Setting up a new cloud environment suitable for producing an RC may be
time consuming, but you are welcome to try. Krisztian -- are you
available next week to help Micah and potentially take over producing
the RC if there are issues?

> Thanks,
> Micah
>
> [1] https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
>
> On Wed, Sep 18, 2019 at 8:29 AM Wes McKinney  wrote:
>>
>> The process should be well documented at this point but there are a
>> number of steps. Note that you need to add your code signing key to
>> the KEYS file in SVN (that's not very hard to do). I think it's fine
>> to hand off the process to others after the VOTE but it would be
>> tricky to have multiple RMs involved with producing the source and
>> binary artifacts for the vote
>>
>> On Tue, Sep 17, 2019 at 10:55 PM Micah Kornfield  
>> wrote:
>> >
>> > SGTM, as well.
>> >
>> > I should have a little bit of time next week if I can help as RM but I have
>> > a couple of concerns:
>> > 1.  In the past I've had trouble downloading and validating releases. I'm a
>> > bit worried, that I might have similar problems doing the necessary 
>> > uploads.
>> > 2.  My internet connection will likely be not great, I don't know if this
>> > would make it even less likely to be successful.
>> >
>> > Does it become problematic if somehow I would have to abandon the process
>> > mid-release?  Is there anyone who could serve as a backup?  Are the steps
>> > well documented?
>> >
>> > Thanks,
>> > Micah
>> >
>> > On Tue, Sep 17, 2019 at 4:25 PM Neal Richardson 
>> > 
>> > wrote:
>> >
>> > > Sounds good to me.
>> > >
>> > > Do we have a release manager yet? Any volunteers?
>> > >
>> > > Neal
>> > >
>> > > On Tue, Sep 17, 2019 at 4:06 PM Wes McKinney  wrote:
>> > >
>> > > > hi all,
>> > > >
>> > > > It looks like we're drawing close to be able to make the 0.15.0
>> > > > release. I would suggest "pencils down" at the end of this week and
>> > > > see if a release candidate can be produced next Monday September 23.
>> > > > Any thoughts or objections?
>> > > >
>> > > > Thanks,
>> > > > Wes
>> > > >
>> > > > On Wed, Sep 11, 2019 at 11:23 AM Wes McKinney 
>> > > wrote:
>> > > > >
>> > > > > hi Eric -- yes, that's correct. I'm planning to amend the Format docs
>> > > > > today regarding the EOS issue and also update the C++ library
>> > > > >
>> > > > > On Wed, Sep 11, 2019 at 11:21 AM Eric Erhardt
>> > > > >  wrote:
>> > > > > >
>> > > > > > I assume the plan is to merge the ARROW-6313-flatbuffer-alignment
>> > > > branch into master before the 0.15 release, correct?
>> > > > > >
>> > > > > > BTW - I believe the C# alignment changes are ready to be merged 
>> > > > > > into
>> > > > the alignment branch -  https://github.com/apache/arrow/pull/5280/
>> > > > > >
>> > > > > > Eric
>> > > > > >
>> > > > > > -Original Message-
>> > > > > > From: Micah Kornfield 
>> > > > > > Sent: Tuesday, September 10, 2019 10:24 PM
>> > > > > > To: Wes McKinney 
>> > > > > > Cc: dev ; niki.lj 
>> > > > > > Subject: Re: Timeline for 0.15.0 release
>> > > > > >
>> > > > > > I should have a little more bandwidth to help with some of the
>> > > > packaging starting tomorrow and going into the weekend.
>> > > > > >
>> > > > > > On Tuesday, September 10, 2019, Wes McKinney 
>> > > > wrote:
>> > > > > >
>> > > > > > > Hi folks,
>> > > > > >

Re: Timeline for 0.15.0 release

2019-09-18 Thread Micah Kornfield
>
> The process should be well documented at this point but there are a
> number of steps.

Is [1] the up-to-date documentation for the release?   Are there
instructions for the adding the code signing Key to SVN?

I will make a go of it.  i will try to mitigate any internet issues by
doing the process for a cloud instance (I assume that isn't a problem?).

Thanks,
Micah

[1]
https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide

On Wed, Sep 18, 2019 at 8:29 AM Wes McKinney  wrote:

> The process should be well documented at this point but there are a
> number of steps. Note that you need to add your code signing key to
> the KEYS file in SVN (that's not very hard to do). I think it's fine
> to hand off the process to others after the VOTE but it would be
> tricky to have multiple RMs involved with producing the source and
> binary artifacts for the vote
>
> On Tue, Sep 17, 2019 at 10:55 PM Micah Kornfield 
> wrote:
> >
> > SGTM, as well.
> >
> > I should have a little bit of time next week if I can help as RM but I
> have
> > a couple of concerns:
> > 1.  In the past I've had trouble downloading and validating releases.
> I'm a
> > bit worried, that I might have similar problems doing the necessary
> uploads.
> > 2.  My internet connection will likely be not great, I don't know if this
> > would make it even less likely to be successful.
> >
> > Does it become problematic if somehow I would have to abandon the process
> > mid-release?  Is there anyone who could serve as a backup?  Are the steps
> > well documented?
> >
> > Thanks,
> > Micah
> >
> > On Tue, Sep 17, 2019 at 4:25 PM Neal Richardson <
> neal.p.richard...@gmail.com>
> > wrote:
> >
> > > Sounds good to me.
> > >
> > > Do we have a release manager yet? Any volunteers?
> > >
> > > Neal
> > >
> > > On Tue, Sep 17, 2019 at 4:06 PM Wes McKinney 
> wrote:
> > >
> > > > hi all,
> > > >
> > > > It looks like we're drawing close to be able to make the 0.15.0
> > > > release. I would suggest "pencils down" at the end of this week and
> > > > see if a release candidate can be produced next Monday September 23.
> > > > Any thoughts or objections?
> > > >
> > > > Thanks,
> > > > Wes
> > > >
> > > > On Wed, Sep 11, 2019 at 11:23 AM Wes McKinney 
> > > wrote:
> > > > >
> > > > > hi Eric -- yes, that's correct. I'm planning to amend the Format
> docs
> > > > > today regarding the EOS issue and also update the C++ library
> > > > >
> > > > > On Wed, Sep 11, 2019 at 11:21 AM Eric Erhardt
> > > > >  wrote:
> > > > > >
> > > > > > I assume the plan is to merge the ARROW-6313-flatbuffer-alignment
> > > > branch into master before the 0.15 release, correct?
> > > > > >
> > > > > > BTW - I believe the C# alignment changes are ready to be merged
> into
> > > > the alignment branch -  https://github.com/apache/arrow/pull/5280/
> > > > > >
> > > > > > Eric
> > > > > >
> > > > > > -Original Message-
> > > > > > From: Micah Kornfield 
> > > > > > Sent: Tuesday, September 10, 2019 10:24 PM
> > > > > > To: Wes McKinney 
> > > > > > Cc: dev ; niki.lj 
> > > > > > Subject: Re: Timeline for 0.15.0 release
> > > > > >
> > > > > > I should have a little more bandwidth to help with some of the
> > > > packaging starting tomorrow and going into the weekend.
> > > > > >
> > > > > > On Tuesday, September 10, 2019, Wes McKinney <
> wesmck...@gmail.com>
> > > > wrote:
> > > > > >
> > > > > > > Hi folks,
> > > > > > >
> > > > > > > With the state of nightly packaging and integration builds
> things
> > > > > > > aren't looking too good for being in release readiness by the
> end
> > > of
> > > > > > > this week but maybe I'm wrong. I'm planning to be working to
> close
> > > as
> > > > > > > many issues as I can and also to help with the ongoing
> alignment
> > > > fixes.
> > > > > > >
> > > > > > > Wes
> > > > > > >
> > > > > > > On Thu, Sep 5, 201

Re: Timeline for 0.15.0 release

2019-09-18 Thread Wes McKinney
The process should be well documented at this point but there are a
number of steps. Note that you need to add your code signing key to
the KEYS file in SVN (that's not very hard to do). I think it's fine
to hand off the process to others after the VOTE but it would be
tricky to have multiple RMs involved with producing the source and
binary artifacts for the vote

On Tue, Sep 17, 2019 at 10:55 PM Micah Kornfield  wrote:
>
> SGTM, as well.
>
> I should have a little bit of time next week if I can help as RM but I have
> a couple of concerns:
> 1.  In the past I've had trouble downloading and validating releases. I'm a
> bit worried, that I might have similar problems doing the necessary uploads.
> 2.  My internet connection will likely be not great, I don't know if this
> would make it even less likely to be successful.
>
> Does it become problematic if somehow I would have to abandon the process
> mid-release?  Is there anyone who could serve as a backup?  Are the steps
> well documented?
>
> Thanks,
> Micah
>
> On Tue, Sep 17, 2019 at 4:25 PM Neal Richardson 
> wrote:
>
> > Sounds good to me.
> >
> > Do we have a release manager yet? Any volunteers?
> >
> > Neal
> >
> > On Tue, Sep 17, 2019 at 4:06 PM Wes McKinney  wrote:
> >
> > > hi all,
> > >
> > > It looks like we're drawing close to be able to make the 0.15.0
> > > release. I would suggest "pencils down" at the end of this week and
> > > see if a release candidate can be produced next Monday September 23.
> > > Any thoughts or objections?
> > >
> > > Thanks,
> > > Wes
> > >
> > > On Wed, Sep 11, 2019 at 11:23 AM Wes McKinney 
> > wrote:
> > > >
> > > > hi Eric -- yes, that's correct. I'm planning to amend the Format docs
> > > > today regarding the EOS issue and also update the C++ library
> > > >
> > > > On Wed, Sep 11, 2019 at 11:21 AM Eric Erhardt
> > > >  wrote:
> > > > >
> > > > > I assume the plan is to merge the ARROW-6313-flatbuffer-alignment
> > > branch into master before the 0.15 release, correct?
> > > > >
> > > > > BTW - I believe the C# alignment changes are ready to be merged into
> > > the alignment branch -  https://github.com/apache/arrow/pull/5280/
> > > > >
> > > > > Eric
> > > > >
> > > > > -Original Message-
> > > > > From: Micah Kornfield 
> > > > > Sent: Tuesday, September 10, 2019 10:24 PM
> > > > > To: Wes McKinney 
> > > > > Cc: dev ; niki.lj 
> > > > > Subject: Re: Timeline for 0.15.0 release
> > > > >
> > > > > I should have a little more bandwidth to help with some of the
> > > packaging starting tomorrow and going into the weekend.
> > > > >
> > > > > On Tuesday, September 10, 2019, Wes McKinney 
> > > wrote:
> > > > >
> > > > > > Hi folks,
> > > > > >
> > > > > > With the state of nightly packaging and integration builds things
> > > > > > aren't looking too good for being in release readiness by the end
> > of
> > > > > > this week but maybe I'm wrong. I'm planning to be working to close
> > as
> > > > > > many issues as I can and also to help with the ongoing alignment
> > > fixes.
> > > > > >
> > > > > > Wes
> > > > > >
> > > > > > On Thu, Sep 5, 2019, 11:07 PM Micah Kornfield <
> > emkornfi...@gmail.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > >> Just for reference [1] has a dashboard of the current issues:
> > > > > >>
> > > > > >>
> > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwi
> > > > > >> ki.apache.org
> > > %2Fconfluence%2Fdisplay%2FARROW%2FArrow%2B0.15.0%2BRelea
> > > > > >> sedata=02%7C01%7CEric.Erhardt%40microsoft.com
> > > %7Ccbead81a42104034
> > > > > >>
> > > a4f308d736678a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6370376
> > > > > >>
> > > 90648216338sdata=0Upux3i%2B9X6f8uanGKSGM5VYxR6c2ADWrxSPi1%2FgbH4
> > > > > >> %3Dreserved=0
> > > > > >>
> > > > > >> On Thu, Sep 5, 2019 at 3:43 PM Wes McKinney 
> > > wrote:
> > > > > >

Re: Timeline for 0.15.0 release

2019-09-17 Thread Micah Kornfield
SGTM, as well.

I should have a little bit of time next week if I can help as RM but I have
a couple of concerns:
1.  In the past I've had trouble downloading and validating releases. I'm a
bit worried, that I might have similar problems doing the necessary uploads.
2.  My internet connection will likely be not great, I don't know if this
would make it even less likely to be successful.

Does it become problematic if somehow I would have to abandon the process
mid-release?  Is there anyone who could serve as a backup?  Are the steps
well documented?

Thanks,
Micah

On Tue, Sep 17, 2019 at 4:25 PM Neal Richardson 
wrote:

> Sounds good to me.
>
> Do we have a release manager yet? Any volunteers?
>
> Neal
>
> On Tue, Sep 17, 2019 at 4:06 PM Wes McKinney  wrote:
>
> > hi all,
> >
> > It looks like we're drawing close to be able to make the 0.15.0
> > release. I would suggest "pencils down" at the end of this week and
> > see if a release candidate can be produced next Monday September 23.
> > Any thoughts or objections?
> >
> > Thanks,
> > Wes
> >
> > On Wed, Sep 11, 2019 at 11:23 AM Wes McKinney 
> wrote:
> > >
> > > hi Eric -- yes, that's correct. I'm planning to amend the Format docs
> > > today regarding the EOS issue and also update the C++ library
> > >
> > > On Wed, Sep 11, 2019 at 11:21 AM Eric Erhardt
> > >  wrote:
> > > >
> > > > I assume the plan is to merge the ARROW-6313-flatbuffer-alignment
> > branch into master before the 0.15 release, correct?
> > > >
> > > > BTW - I believe the C# alignment changes are ready to be merged into
> > the alignment branch -  https://github.com/apache/arrow/pull/5280/
> > > >
> > > > Eric
> > > >
> > > > -Original Message-
> > > > From: Micah Kornfield 
> > > > Sent: Tuesday, September 10, 2019 10:24 PM
> > > > To: Wes McKinney 
> > > > Cc: dev ; niki.lj 
> > > > Subject: Re: Timeline for 0.15.0 release
> > > >
> > > > I should have a little more bandwidth to help with some of the
> > packaging starting tomorrow and going into the weekend.
> > > >
> > > > On Tuesday, September 10, 2019, Wes McKinney 
> > wrote:
> > > >
> > > > > Hi folks,
> > > > >
> > > > > With the state of nightly packaging and integration builds things
> > > > > aren't looking too good for being in release readiness by the end
> of
> > > > > this week but maybe I'm wrong. I'm planning to be working to close
> as
> > > > > many issues as I can and also to help with the ongoing alignment
> > fixes.
> > > > >
> > > > > Wes
> > > > >
> > > > > On Thu, Sep 5, 2019, 11:07 PM Micah Kornfield <
> emkornfi...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > >> Just for reference [1] has a dashboard of the current issues:
> > > > >>
> > > > >>
> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwi
> > > > >> ki.apache.org
> > %2Fconfluence%2Fdisplay%2FARROW%2FArrow%2B0.15.0%2BRelea
> > > > >> sedata=02%7C01%7CEric.Erhardt%40microsoft.com
> > %7Ccbead81a42104034
> > > > >>
> > a4f308d736678a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6370376
> > > > >>
> > 90648216338sdata=0Upux3i%2B9X6f8uanGKSGM5VYxR6c2ADWrxSPi1%2FgbH4
> > > > >> %3Dreserved=0
> > > > >>
> > > > >> On Thu, Sep 5, 2019 at 3:43 PM Wes McKinney 
> > wrote:
> > > > >>
> > > > >>> hi all,
> > > > >>>
> > > > >>> It doesn't seem like we're going to be in a position to release
> at
> > > > >>> the beginning of next week. I hope that one more week of work (or
> > > > >>> less) will be enough to get us there. Aside from merging the
> > > > >>> alignment changes, we need to make sure that our packaging jobs
> > > > >>> required for the release candidate are all working.
> > > > >>>
> > > > >>> If folks could remove issues from the 0.15.0 backlog that they
> > don't
> > > > >>> think they will finish by end of next week that would help focus
> > > > >>> efforts (there are currently 78 issues in 0.15.0 still). I am
> > > > >>> looking to

Re: Timeline for 0.15.0 release

2019-09-17 Thread Neal Richardson
Sounds good to me.

Do we have a release manager yet? Any volunteers?

Neal

On Tue, Sep 17, 2019 at 4:06 PM Wes McKinney  wrote:

> hi all,
>
> It looks like we're drawing close to be able to make the 0.15.0
> release. I would suggest "pencils down" at the end of this week and
> see if a release candidate can be produced next Monday September 23.
> Any thoughts or objections?
>
> Thanks,
> Wes
>
> On Wed, Sep 11, 2019 at 11:23 AM Wes McKinney  wrote:
> >
> > hi Eric -- yes, that's correct. I'm planning to amend the Format docs
> > today regarding the EOS issue and also update the C++ library
> >
> > On Wed, Sep 11, 2019 at 11:21 AM Eric Erhardt
> >  wrote:
> > >
> > > I assume the plan is to merge the ARROW-6313-flatbuffer-alignment
> branch into master before the 0.15 release, correct?
> > >
> > > BTW - I believe the C# alignment changes are ready to be merged into
> the alignment branch -  https://github.com/apache/arrow/pull/5280/
> > >
> > > Eric
> > >
> > > -Original Message-----
> > > From: Micah Kornfield 
> > > Sent: Tuesday, September 10, 2019 10:24 PM
> > > To: Wes McKinney 
> > > Cc: dev ; niki.lj 
> > > Subject: Re: Timeline for 0.15.0 release
> > >
> > > I should have a little more bandwidth to help with some of the
> packaging starting tomorrow and going into the weekend.
> > >
> > > On Tuesday, September 10, 2019, Wes McKinney 
> wrote:
> > >
> > > > Hi folks,
> > > >
> > > > With the state of nightly packaging and integration builds things
> > > > aren't looking too good for being in release readiness by the end of
> > > > this week but maybe I'm wrong. I'm planning to be working to close as
> > > > many issues as I can and also to help with the ongoing alignment
> fixes.
> > > >
> > > > Wes
> > > >
> > > > On Thu, Sep 5, 2019, 11:07 PM Micah Kornfield  >
> > > > wrote:
> > > >
> > > >> Just for reference [1] has a dashboard of the current issues:
> > > >>
> > > >>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwi
> > > >> ki.apache.org
> %2Fconfluence%2Fdisplay%2FARROW%2FArrow%2B0.15.0%2BRelea
> > > >> sedata=02%7C01%7CEric.Erhardt%40microsoft.com
> %7Ccbead81a42104034
> > > >>
> a4f308d736678a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6370376
> > > >>
> 90648216338sdata=0Upux3i%2B9X6f8uanGKSGM5VYxR6c2ADWrxSPi1%2FgbH4
> > > >> %3Dreserved=0
> > > >>
> > > >> On Thu, Sep 5, 2019 at 3:43 PM Wes McKinney 
> wrote:
> > > >>
> > > >>> hi all,
> > > >>>
> > > >>> It doesn't seem like we're going to be in a position to release at
> > > >>> the beginning of next week. I hope that one more week of work (or
> > > >>> less) will be enough to get us there. Aside from merging the
> > > >>> alignment changes, we need to make sure that our packaging jobs
> > > >>> required for the release candidate are all working.
> > > >>>
> > > >>> If folks could remove issues from the 0.15.0 backlog that they
> don't
> > > >>> think they will finish by end of next week that would help focus
> > > >>> efforts (there are currently 78 issues in 0.15.0 still). I am
> > > >>> looking to tackle a few small features related to dictionaries
> while
> > > >>> the release window is still open.
> > > >>>
> > > >>> - Wes
> > > >>>
> > > >>> On Tue, Aug 27, 2019 at 3:48 PM Wes McKinney 
> > > >>> wrote:
> > > >>> >
> > > >>> > hi,
> > > >>> >
> > > >>> > I think we should try to release the week of September 9, so
> > > >>> > development work should be completed by end of next week.
> > > >>> >
> > > >>> > Does that seem reasonable?
> > > >>> >
> > > >>> > I plan to get up a patch for the protocol alignment changes for
> > > >>> > C++ in the next couple of days -- I think that getting the
> > > >>> > alignment work done is the main barrier to releasing.
> > > >>> >
> > > >>> > Thanks
> > > >>> &

Re: Timeline for 0.15.0 release

2019-09-17 Thread Wes McKinney
hi all,

It looks like we're drawing close to be able to make the 0.15.0
release. I would suggest "pencils down" at the end of this week and
see if a release candidate can be produced next Monday September 23.
Any thoughts or objections?

Thanks,
Wes

On Wed, Sep 11, 2019 at 11:23 AM Wes McKinney  wrote:
>
> hi Eric -- yes, that's correct. I'm planning to amend the Format docs
> today regarding the EOS issue and also update the C++ library
>
> On Wed, Sep 11, 2019 at 11:21 AM Eric Erhardt
>  wrote:
> >
> > I assume the plan is to merge the ARROW-6313-flatbuffer-alignment branch 
> > into master before the 0.15 release, correct?
> >
> > BTW - I believe the C# alignment changes are ready to be merged into the 
> > alignment branch -  https://github.com/apache/arrow/pull/5280/
> >
> > Eric
> >
> > -Original Message-
> > From: Micah Kornfield 
> > Sent: Tuesday, September 10, 2019 10:24 PM
> > To: Wes McKinney 
> > Cc: dev ; niki.lj 
> > Subject: Re: Timeline for 0.15.0 release
> >
> > I should have a little more bandwidth to help with some of the packaging 
> > starting tomorrow and going into the weekend.
> >
> > On Tuesday, September 10, 2019, Wes McKinney  wrote:
> >
> > > Hi folks,
> > >
> > > With the state of nightly packaging and integration builds things
> > > aren't looking too good for being in release readiness by the end of
> > > this week but maybe I'm wrong. I'm planning to be working to close as
> > > many issues as I can and also to help with the ongoing alignment fixes.
> > >
> > > Wes
> > >
> > > On Thu, Sep 5, 2019, 11:07 PM Micah Kornfield 
> > > wrote:
> > >
> > >> Just for reference [1] has a dashboard of the current issues:
> > >>
> > >> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwi
> > >> ki.apache.org%2Fconfluence%2Fdisplay%2FARROW%2FArrow%2B0.15.0%2BRelea
> > >> sedata=02%7C01%7CEric.Erhardt%40microsoft.com%7Ccbead81a42104034
> > >> a4f308d736678a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6370376
> > >> 90648216338sdata=0Upux3i%2B9X6f8uanGKSGM5VYxR6c2ADWrxSPi1%2FgbH4
> > >> %3Dreserved=0
> > >>
> > >> On Thu, Sep 5, 2019 at 3:43 PM Wes McKinney  wrote:
> > >>
> > >>> hi all,
> > >>>
> > >>> It doesn't seem like we're going to be in a position to release at
> > >>> the beginning of next week. I hope that one more week of work (or
> > >>> less) will be enough to get us there. Aside from merging the
> > >>> alignment changes, we need to make sure that our packaging jobs
> > >>> required for the release candidate are all working.
> > >>>
> > >>> If folks could remove issues from the 0.15.0 backlog that they don't
> > >>> think they will finish by end of next week that would help focus
> > >>> efforts (there are currently 78 issues in 0.15.0 still). I am
> > >>> looking to tackle a few small features related to dictionaries while
> > >>> the release window is still open.
> > >>>
> > >>> - Wes
> > >>>
> > >>> On Tue, Aug 27, 2019 at 3:48 PM Wes McKinney 
> > >>> wrote:
> > >>> >
> > >>> > hi,
> > >>> >
> > >>> > I think we should try to release the week of September 9, so
> > >>> > development work should be completed by end of next week.
> > >>> >
> > >>> > Does that seem reasonable?
> > >>> >
> > >>> > I plan to get up a patch for the protocol alignment changes for
> > >>> > C++ in the next couple of days -- I think that getting the
> > >>> > alignment work done is the main barrier to releasing.
> > >>> >
> > >>> > Thanks
> > >>> > Wes
> > >>> >
> > >>> > On Mon, Aug 19, 2019 at 12:25 PM Ji Liu
> > >>> > 
> > >>> wrote:
> > >>> > >
> > >>> > > Hi, Wes, on the java side, I can think of several bugs that need
> > >>> > > to
> > >>> be fixed or reminded.
> > >>> > >
> > >>> > > i. ARROW-6040: Dictionary entries are required in IPC streams
> > >>> > > even
> > >>> when empty[1]
> > >>> > > This one is under re

Re: Timeline for 0.15.0 release

2019-09-11 Thread Wes McKinney
hi Eric -- yes, that's correct. I'm planning to amend the Format docs
today regarding the EOS issue and also update the C++ library

On Wed, Sep 11, 2019 at 11:21 AM Eric Erhardt
 wrote:
>
> I assume the plan is to merge the ARROW-6313-flatbuffer-alignment branch into 
> master before the 0.15 release, correct?
>
> BTW - I believe the C# alignment changes are ready to be merged into the 
> alignment branch -  https://github.com/apache/arrow/pull/5280/
>
> Eric
>
> -Original Message-
> From: Micah Kornfield 
> Sent: Tuesday, September 10, 2019 10:24 PM
> To: Wes McKinney 
> Cc: dev ; niki.lj 
> Subject: Re: Timeline for 0.15.0 release
>
> I should have a little more bandwidth to help with some of the packaging 
> starting tomorrow and going into the weekend.
>
> On Tuesday, September 10, 2019, Wes McKinney  wrote:
>
> > Hi folks,
> >
> > With the state of nightly packaging and integration builds things
> > aren't looking too good for being in release readiness by the end of
> > this week but maybe I'm wrong. I'm planning to be working to close as
> > many issues as I can and also to help with the ongoing alignment fixes.
> >
> > Wes
> >
> > On Thu, Sep 5, 2019, 11:07 PM Micah Kornfield 
> > wrote:
> >
> >> Just for reference [1] has a dashboard of the current issues:
> >>
> >> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwi
> >> ki.apache.org%2Fconfluence%2Fdisplay%2FARROW%2FArrow%2B0.15.0%2BRelea
> >> sedata=02%7C01%7CEric.Erhardt%40microsoft.com%7Ccbead81a42104034
> >> a4f308d736678a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6370376
> >> 90648216338sdata=0Upux3i%2B9X6f8uanGKSGM5VYxR6c2ADWrxSPi1%2FgbH4
> >> %3Dreserved=0
> >>
> >> On Thu, Sep 5, 2019 at 3:43 PM Wes McKinney  wrote:
> >>
> >>> hi all,
> >>>
> >>> It doesn't seem like we're going to be in a position to release at
> >>> the beginning of next week. I hope that one more week of work (or
> >>> less) will be enough to get us there. Aside from merging the
> >>> alignment changes, we need to make sure that our packaging jobs
> >>> required for the release candidate are all working.
> >>>
> >>> If folks could remove issues from the 0.15.0 backlog that they don't
> >>> think they will finish by end of next week that would help focus
> >>> efforts (there are currently 78 issues in 0.15.0 still). I am
> >>> looking to tackle a few small features related to dictionaries while
> >>> the release window is still open.
> >>>
> >>> - Wes
> >>>
> >>> On Tue, Aug 27, 2019 at 3:48 PM Wes McKinney 
> >>> wrote:
> >>> >
> >>> > hi,
> >>> >
> >>> > I think we should try to release the week of September 9, so
> >>> > development work should be completed by end of next week.
> >>> >
> >>> > Does that seem reasonable?
> >>> >
> >>> > I plan to get up a patch for the protocol alignment changes for
> >>> > C++ in the next couple of days -- I think that getting the
> >>> > alignment work done is the main barrier to releasing.
> >>> >
> >>> > Thanks
> >>> > Wes
> >>> >
> >>> > On Mon, Aug 19, 2019 at 12:25 PM Ji Liu
> >>> > 
> >>> wrote:
> >>> > >
> >>> > > Hi, Wes, on the java side, I can think of several bugs that need
> >>> > > to
> >>> be fixed or reminded.
> >>> > >
> >>> > > i. ARROW-6040: Dictionary entries are required in IPC streams
> >>> > > even
> >>> when empty[1]
> >>> > > This one is under review now, however through this PR we find
> >>> > > that
> >>> there seems a bug in java reading and writing dictionaries in IPC
> >>> which is Inconsistent with spec[2] since it assumes all dictionaries
> >>> are at the start of stream (see details in PR comments,  and this
> >>> fix may not catch up with version 0.15). @Micah Kornfield
> >>> > >
> >>> > > ii. ARROW-1875: Write 64-bit ints as strings in integration test
> >>> JSON files[3]
> >>> > > Java side code already checked in, other implementations seems not.
> >>> > >
> >>> > > iii. ARROW-6202: OutOfMemory in JdbcAdapter[4] Caused by trying
> >

RE: Timeline for 0.15.0 release

2019-09-11 Thread Eric Erhardt
I assume the plan is to merge the ARROW-6313-flatbuffer-alignment branch into 
master before the 0.15 release, correct?

BTW - I believe the C# alignment changes are ready to be merged into the 
alignment branch -  https://github.com/apache/arrow/pull/5280/ 

Eric

-Original Message-
From: Micah Kornfield  
Sent: Tuesday, September 10, 2019 10:24 PM
To: Wes McKinney 
Cc: dev ; niki.lj 
Subject: Re: Timeline for 0.15.0 release

I should have a little more bandwidth to help with some of the packaging 
starting tomorrow and going into the weekend.

On Tuesday, September 10, 2019, Wes McKinney  wrote:

> Hi folks,
>
> With the state of nightly packaging and integration builds things 
> aren't looking too good for being in release readiness by the end of 
> this week but maybe I'm wrong. I'm planning to be working to close as 
> many issues as I can and also to help with the ongoing alignment fixes.
>
> Wes
>
> On Thu, Sep 5, 2019, 11:07 PM Micah Kornfield 
> wrote:
>
>> Just for reference [1] has a dashboard of the current issues:
>>
>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwi
>> ki.apache.org%2Fconfluence%2Fdisplay%2FARROW%2FArrow%2B0.15.0%2BRelea
>> sedata=02%7C01%7CEric.Erhardt%40microsoft.com%7Ccbead81a42104034
>> a4f308d736678a45%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6370376
>> 90648216338sdata=0Upux3i%2B9X6f8uanGKSGM5VYxR6c2ADWrxSPi1%2FgbH4
>> %3Dreserved=0
>>
>> On Thu, Sep 5, 2019 at 3:43 PM Wes McKinney  wrote:
>>
>>> hi all,
>>>
>>> It doesn't seem like we're going to be in a position to release at 
>>> the beginning of next week. I hope that one more week of work (or 
>>> less) will be enough to get us there. Aside from merging the 
>>> alignment changes, we need to make sure that our packaging jobs 
>>> required for the release candidate are all working.
>>>
>>> If folks could remove issues from the 0.15.0 backlog that they don't 
>>> think they will finish by end of next week that would help focus 
>>> efforts (there are currently 78 issues in 0.15.0 still). I am 
>>> looking to tackle a few small features related to dictionaries while 
>>> the release window is still open.
>>>
>>> - Wes
>>>
>>> On Tue, Aug 27, 2019 at 3:48 PM Wes McKinney 
>>> wrote:
>>> >
>>> > hi,
>>> >
>>> > I think we should try to release the week of September 9, so 
>>> > development work should be completed by end of next week.
>>> >
>>> > Does that seem reasonable?
>>> >
>>> > I plan to get up a patch for the protocol alignment changes for 
>>> > C++ in the next couple of days -- I think that getting the 
>>> > alignment work done is the main barrier to releasing.
>>> >
>>> > Thanks
>>> > Wes
>>> >
>>> > On Mon, Aug 19, 2019 at 12:25 PM Ji Liu 
>>> > 
>>> wrote:
>>> > >
>>> > > Hi, Wes, on the java side, I can think of several bugs that need 
>>> > > to
>>> be fixed or reminded.
>>> > >
>>> > > i. ARROW-6040: Dictionary entries are required in IPC streams 
>>> > > even
>>> when empty[1]
>>> > > This one is under review now, however through this PR we find 
>>> > > that
>>> there seems a bug in java reading and writing dictionaries in IPC 
>>> which is Inconsistent with spec[2] since it assumes all dictionaries 
>>> are at the start of stream (see details in PR comments,  and this 
>>> fix may not catch up with version 0.15). @Micah Kornfield
>>> > >
>>> > > ii. ARROW-1875: Write 64-bit ints as strings in integration test
>>> JSON files[3]
>>> > > Java side code already checked in, other implementations seems not.
>>> > >
>>> > > iii. ARROW-6202: OutOfMemory in JdbcAdapter[4] Caused by trying 
>>> > > to load all records in one contiguous batch, fixed
>>> by providing iterator API for iteratively reading in ARROW-6219[5].
>>> > >
>>> > > Thanks,
>>> > > Ji Liu
>>> > >
>>> > > [1] 
>>> > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%
>>> > > 2Fgithub.com%2Fapache%2Farrow%2Fpull%2F4960data=02%7C01%7CE
>>> > > ric.Erhardt%40microsoft.com%7Ccbead81a42104034a4f308d736678a45%7
>>> > > C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637037690648216338
>>> > > mp;sdata=eDF%2FAsJ

Re: Timeline for 0.15.0 release

2019-09-10 Thread Micah Kornfield
I should have a little more bandwidth to help with some of the packaging
starting tomorrow and going into the weekend.

On Tuesday, September 10, 2019, Wes McKinney  wrote:

> Hi folks,
>
> With the state of nightly packaging and integration builds things aren't
> looking too good for being in release readiness by the end of this week but
> maybe I'm wrong. I'm planning to be working to close as many issues as I
> can and also to help with the ongoing alignment fixes.
>
> Wes
>
> On Thu, Sep 5, 2019, 11:07 PM Micah Kornfield 
> wrote:
>
>> Just for reference [1] has a dashboard of the current issues:
>>
>> https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.15.0+Release
>>
>> On Thu, Sep 5, 2019 at 3:43 PM Wes McKinney  wrote:
>>
>>> hi all,
>>>
>>> It doesn't seem like we're going to be in a position to release at the
>>> beginning of next week. I hope that one more week of work (or less)
>>> will be enough to get us there. Aside from merging the alignment
>>> changes, we need to make sure that our packaging jobs required for the
>>> release candidate are all working.
>>>
>>> If folks could remove issues from the 0.15.0 backlog that they don't
>>> think they will finish by end of next week that would help focus
>>> efforts (there are currently 78 issues in 0.15.0 still). I am looking
>>> to tackle a few small features related to dictionaries while the
>>> release window is still open.
>>>
>>> - Wes
>>>
>>> On Tue, Aug 27, 2019 at 3:48 PM Wes McKinney 
>>> wrote:
>>> >
>>> > hi,
>>> >
>>> > I think we should try to release the week of September 9, so
>>> > development work should be completed by end of next week.
>>> >
>>> > Does that seem reasonable?
>>> >
>>> > I plan to get up a patch for the protocol alignment changes for C++ in
>>> > the next couple of days -- I think that getting the alignment work
>>> > done is the main barrier to releasing.
>>> >
>>> > Thanks
>>> > Wes
>>> >
>>> > On Mon, Aug 19, 2019 at 12:25 PM Ji Liu 
>>> wrote:
>>> > >
>>> > > Hi, Wes, on the java side, I can think of several bugs that need to
>>> be fixed or reminded.
>>> > >
>>> > > i. ARROW-6040: Dictionary entries are required in IPC streams even
>>> when empty[1]
>>> > > This one is under review now, however through this PR we find that
>>> there seems a bug in java reading and writing dictionaries in IPC which is
>>> Inconsistent with spec[2] since it assumes all dictionaries are at the
>>> start of stream (see details in PR comments,  and this fix may not catch up
>>> with version 0.15). @Micah Kornfield
>>> > >
>>> > > ii. ARROW-1875: Write 64-bit ints as strings in integration test
>>> JSON files[3]
>>> > > Java side code already checked in, other implementations seems not.
>>> > >
>>> > > iii. ARROW-6202: OutOfMemory in JdbcAdapter[4]
>>> > > Caused by trying to load all records in one contiguous batch, fixed
>>> by providing iterator API for iteratively reading in ARROW-6219[5].
>>> > >
>>> > > Thanks,
>>> > > Ji Liu
>>> > >
>>> > > [1] https://github.com/apache/arrow/pull/4960
>>> > > [2] https://arrow.apache.org/docs/ipc.html
>>> > > [3] https://issues.apache.org/jira/browse/ARROW-1875
>>> > > [4] https://issues.apache.org/jira/browse/ARROW-6202[5]
>>> https://issues.apache.org/jira/browse/ARROW-6219
>>> > >
>>> > >
>>> > >
>>> > > --
>>> > > From:Wes McKinney 
>>> > > Send Time:2019年8月19日(星期一) 23:03
>>> > > To:dev 
>>> > > Subject:Re: Timeline for 0.15.0 release
>>> > >
>>> > > I'm going to work some on organizing the 0.15.0 backlog some this
>>> > > week, if anyone wants to help with grooming (particularly for
>>> > > languages other than C++/Python where I'm focusing) that would be
>>> > > helpful. There have been almost 500 JIRA issues opened since the
>>> > > 0.14.0 release, so we should make sure to check whether there's any
>>> > > regressions or other serious bugs that we should try to fix for
>>> > > 0.15.0.
>>> > >
>>> > > On Thu, Aug 15, 2019 at 6:23 PM Wes McKinney 
>>> wrote:
>>> > > >
>>> > > > The Windows wheel issue in 0.14.1 seems to be
>>> > > >
>>> > > > https://issues.apache.org/jira/browse/ARROW-6015
>>> > > >
>>> > > > I think the root cause could be the Windows changes in
>>> > > >
>>> > > > https://github.com/apache/arrow/commit/
>>> 223ae744cc2a12c60cecb5db593263a03c13f85a
>>> > > >
>>> > > > I would be appreciative if a volunteer would look into what was
>>> wrong
>>> > > > with the 0.14.1 wheels on Windows. Otherwise 0.15.0 Windows wheels
>>> > > > will be broken, too
>>> > > >
>>> > > > The bad wheels can be found at
>>> > > >
>>> > > > https://bintray.com/apache/arrow/python#files/python%2F0.14.1
>>> > > >
>>> > > > On Thu, Aug 15, 2019 at 1:28 PM Antoine Pitrou <
>>> solip...@pitrou.net> wrote:
>>> > > > >
>>> > > > > On Thu, 15 Aug 2019 11:17:07 -0700
>>> > > > > Micah Kornfield  wrote:
>>> > > > > > >
>>> > > > > > > In C++ they are
>>> > > > > > > independent, we could have 32-bit array lengths and
>>> 

Re: Timeline for 0.15.0 release

2019-09-10 Thread Wes McKinney
Hi folks,

With the state of nightly packaging and integration builds things aren't
looking too good for being in release readiness by the end of this week but
maybe I'm wrong. I'm planning to be working to close as many issues as I
can and also to help with the ongoing alignment fixes.

Wes

On Thu, Sep 5, 2019, 11:07 PM Micah Kornfield  wrote:

> Just for reference [1] has a dashboard of the current issues:
>
> https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.15.0+Release
>
> On Thu, Sep 5, 2019 at 3:43 PM Wes McKinney  wrote:
>
>> hi all,
>>
>> It doesn't seem like we're going to be in a position to release at the
>> beginning of next week. I hope that one more week of work (or less)
>> will be enough to get us there. Aside from merging the alignment
>> changes, we need to make sure that our packaging jobs required for the
>> release candidate are all working.
>>
>> If folks could remove issues from the 0.15.0 backlog that they don't
>> think they will finish by end of next week that would help focus
>> efforts (there are currently 78 issues in 0.15.0 still). I am looking
>> to tackle a few small features related to dictionaries while the
>> release window is still open.
>>
>> - Wes
>>
>> On Tue, Aug 27, 2019 at 3:48 PM Wes McKinney  wrote:
>> >
>> > hi,
>> >
>> > I think we should try to release the week of September 9, so
>> > development work should be completed by end of next week.
>> >
>> > Does that seem reasonable?
>> >
>> > I plan to get up a patch for the protocol alignment changes for C++ in
>> > the next couple of days -- I think that getting the alignment work
>> > done is the main barrier to releasing.
>> >
>> > Thanks
>> > Wes
>> >
>> > On Mon, Aug 19, 2019 at 12:25 PM Ji Liu 
>> wrote:
>> > >
>> > > Hi, Wes, on the java side, I can think of several bugs that need to
>> be fixed or reminded.
>> > >
>> > > i. ARROW-6040: Dictionary entries are required in IPC streams even
>> when empty[1]
>> > > This one is under review now, however through this PR we find that
>> there seems a bug in java reading and writing dictionaries in IPC which is
>> Inconsistent with spec[2] since it assumes all dictionaries are at the
>> start of stream (see details in PR comments,  and this fix may not catch up
>> with version 0.15). @Micah Kornfield
>> > >
>> > > ii. ARROW-1875: Write 64-bit ints as strings in integration test JSON
>> files[3]
>> > > Java side code already checked in, other implementations seems not.
>> > >
>> > > iii. ARROW-6202: OutOfMemory in JdbcAdapter[4]
>> > > Caused by trying to load all records in one contiguous batch, fixed
>> by providing iterator API for iteratively reading in ARROW-6219[5].
>> > >
>> > > Thanks,
>> > > Ji Liu
>> > >
>> > > [1] https://github.com/apache/arrow/pull/4960
>> > > [2] https://arrow.apache.org/docs/ipc.html
>> > > [3] https://issues.apache.org/jira/browse/ARROW-1875
>> > > [4] https://issues.apache.org/jira/browse/ARROW-6202[5]
>> https://issues.apache.org/jira/browse/ARROW-6219
>> > >
>> > >
>> > >
>> > > --
>> > > From:Wes McKinney 
>> > > Send Time:2019年8月19日(星期一) 23:03
>> > > To:dev 
>> > > Subject:Re: Timeline for 0.15.0 release
>> > >
>> > > I'm going to work some on organizing the 0.15.0 backlog some this
>> > > week, if anyone wants to help with grooming (particularly for
>> > > languages other than C++/Python where I'm focusing) that would be
>> > > helpful. There have been almost 500 JIRA issues opened since the
>> > > 0.14.0 release, so we should make sure to check whether there's any
>> > > regressions or other serious bugs that we should try to fix for
>> > > 0.15.0.
>> > >
>> > > On Thu, Aug 15, 2019 at 6:23 PM Wes McKinney 
>> wrote:
>> > > >
>> > > > The Windows wheel issue in 0.14.1 seems to be
>> > > >
>> > > > https://issues.apache.org/jira/browse/ARROW-6015
>> > > >
>> > > > I think the root cause could be the Windows changes in
>> > > >
>> > > >
>> https://github.com/apache/arrow/commit/223ae744cc2a12c60cecb5db593263a03c13f85a
>> > > >
>> > > > I would be appreciative if a volunteer would look into what was
>> wrong
>> > > > with the 0.14.1 wheels on Windows. Otherwise 0.15.0 Windows wheels
>> > > > will be broken, too
>> > > >
>> > > > The bad wheels can be found at
>> > > >
>> > > > https://bintray.com/apache/arrow/python#files/python%2F0.14.1
>> > > >
>> > > > On Thu, Aug 15, 2019 at 1:28 PM Antoine Pitrou 
>> wrote:
>> > > > >
>> > > > > On Thu, 15 Aug 2019 11:17:07 -0700
>> > > > > Micah Kornfield  wrote:
>> > > > > > >
>> > > > > > > In C++ they are
>> > > > > > > independent, we could have 32-bit array lengths and
>> variable-length
>> > > > > > > types with 64-bit offsets if we wanted (we just wouldn't be
>> able to
>> > > > > > > have a List child with more than INT32_MAX elements).
>> > > > > >
>> > > > > > I think the point is we could do this in C++ but we don't.  I'm
>> not sure we
>> > > > > > would have introduced the "Large" types if we did.
>> > > 

Re: Timeline for 0.15.0 release

2019-09-05 Thread Micah Kornfield
Just for reference [1] has a dashboard of the current issues:

https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.15.0+Release

On Thu, Sep 5, 2019 at 3:43 PM Wes McKinney  wrote:

> hi all,
>
> It doesn't seem like we're going to be in a position to release at the
> beginning of next week. I hope that one more week of work (or less)
> will be enough to get us there. Aside from merging the alignment
> changes, we need to make sure that our packaging jobs required for the
> release candidate are all working.
>
> If folks could remove issues from the 0.15.0 backlog that they don't
> think they will finish by end of next week that would help focus
> efforts (there are currently 78 issues in 0.15.0 still). I am looking
> to tackle a few small features related to dictionaries while the
> release window is still open.
>
> - Wes
>
> On Tue, Aug 27, 2019 at 3:48 PM Wes McKinney  wrote:
> >
> > hi,
> >
> > I think we should try to release the week of September 9, so
> > development work should be completed by end of next week.
> >
> > Does that seem reasonable?
> >
> > I plan to get up a patch for the protocol alignment changes for C++ in
> > the next couple of days -- I think that getting the alignment work
> > done is the main barrier to releasing.
> >
> > Thanks
> > Wes
> >
> > On Mon, Aug 19, 2019 at 12:25 PM Ji Liu 
> wrote:
> > >
> > > Hi, Wes, on the java side, I can think of several bugs that need to be
> fixed or reminded.
> > >
> > > i. ARROW-6040: Dictionary entries are required in IPC streams even
> when empty[1]
> > > This one is under review now, however through this PR we find that
> there seems a bug in java reading and writing dictionaries in IPC which is
> Inconsistent with spec[2] since it assumes all dictionaries are at the
> start of stream (see details in PR comments,  and this fix may not catch up
> with version 0.15). @Micah Kornfield
> > >
> > > ii. ARROW-1875: Write 64-bit ints as strings in integration test JSON
> files[3]
> > > Java side code already checked in, other implementations seems not.
> > >
> > > iii. ARROW-6202: OutOfMemory in JdbcAdapter[4]
> > > Caused by trying to load all records in one contiguous batch, fixed by
> providing iterator API for iteratively reading in ARROW-6219[5].
> > >
> > > Thanks,
> > > Ji Liu
> > >
> > > [1] https://github.com/apache/arrow/pull/4960
> > > [2] https://arrow.apache.org/docs/ipc.html
> > > [3] https://issues.apache.org/jira/browse/ARROW-1875
> > > [4] https://issues.apache.org/jira/browse/ARROW-6202[5]
> https://issues.apache.org/jira/browse/ARROW-6219
> > >
> > >
> > >
> > > --
> > > From:Wes McKinney 
> > > Send Time:2019年8月19日(星期一) 23:03
> > > To:dev 
> > > Subject:Re: Timeline for 0.15.0 release
> > >
> > > I'm going to work some on organizing the 0.15.0 backlog some this
> > > week, if anyone wants to help with grooming (particularly for
> > > languages other than C++/Python where I'm focusing) that would be
> > > helpful. There have been almost 500 JIRA issues opened since the
> > > 0.14.0 release, so we should make sure to check whether there's any
> > > regressions or other serious bugs that we should try to fix for
> > > 0.15.0.
> > >
> > > On Thu, Aug 15, 2019 at 6:23 PM Wes McKinney 
> wrote:
> > > >
> > > > The Windows wheel issue in 0.14.1 seems to be
> > > >
> > > > https://issues.apache.org/jira/browse/ARROW-6015
> > > >
> > > > I think the root cause could be the Windows changes in
> > > >
> > > >
> https://github.com/apache/arrow/commit/223ae744cc2a12c60cecb5db593263a03c13f85a
> > > >
> > > > I would be appreciative if a volunteer would look into what was wrong
> > > > with the 0.14.1 wheels on Windows. Otherwise 0.15.0 Windows wheels
> > > > will be broken, too
> > > >
> > > > The bad wheels can be found at
> > > >
> > > > https://bintray.com/apache/arrow/python#files/python%2F0.14.1
> > > >
> > > > On Thu, Aug 15, 2019 at 1:28 PM Antoine Pitrou 
> wrote:
> > > > >
> > > > > On Thu, 15 Aug 2019 11:17:07 -0700
> > > > > Micah Kornfield  wrote:
> > > > > > >
> > > > > > > In C++ they are
> > > > > > > independent, we could have 32-bit array lengths and
> variable-length
> > > > > > > types with 64-bit offsets if we wanted (we just wouldn't be
> able to
> > > > > > > have a List child with more than INT32_MAX elements).
> > > > > >
> > > > > > I think the point is we could do this in C++ but we don't.  I'm
> not sure we
> > > > > > would have introduced the "Large" types if we did.
> > > > >
> > > > > 64-bit offsets take twice as much space as 32-bit offsets, so if
> you're
> > > > > storing lots of small-ish lists or strings, 32-bit offsets are
> > > > > preferrable.  So even with 64-bit array lengths from the start it
> would
> > > > > still be beneficial to have types with 32-bit offsets.
> > > > >
> > > > > > Going with the limited address space in Java and calling it a
> reference
> > > > > > implementation seems suboptimal. If a consumer uses a 

Re: Timeline for 0.15.0 release

2019-09-05 Thread Wes McKinney
hi all,

It doesn't seem like we're going to be in a position to release at the
beginning of next week. I hope that one more week of work (or less)
will be enough to get us there. Aside from merging the alignment
changes, we need to make sure that our packaging jobs required for the
release candidate are all working.

If folks could remove issues from the 0.15.0 backlog that they don't
think they will finish by end of next week that would help focus
efforts (there are currently 78 issues in 0.15.0 still). I am looking
to tackle a few small features related to dictionaries while the
release window is still open.

- Wes

On Tue, Aug 27, 2019 at 3:48 PM Wes McKinney  wrote:
>
> hi,
>
> I think we should try to release the week of September 9, so
> development work should be completed by end of next week.
>
> Does that seem reasonable?
>
> I plan to get up a patch for the protocol alignment changes for C++ in
> the next couple of days -- I think that getting the alignment work
> done is the main barrier to releasing.
>
> Thanks
> Wes
>
> On Mon, Aug 19, 2019 at 12:25 PM Ji Liu  wrote:
> >
> > Hi, Wes, on the java side, I can think of several bugs that need to be 
> > fixed or reminded.
> >
> > i. ARROW-6040: Dictionary entries are required in IPC streams even when 
> > empty[1]
> > This one is under review now, however through this PR we find that there 
> > seems a bug in java reading and writing dictionaries in IPC which is 
> > Inconsistent with spec[2] since it assumes all dictionaries are at the 
> > start of stream (see details in PR comments,  and this fix may not catch up 
> > with version 0.15). @Micah Kornfield
> >
> > ii. ARROW-1875: Write 64-bit ints as strings in integration test JSON 
> > files[3]
> > Java side code already checked in, other implementations seems not.
> >
> > iii. ARROW-6202: OutOfMemory in JdbcAdapter[4]
> > Caused by trying to load all records in one contiguous batch, fixed by 
> > providing iterator API for iteratively reading in ARROW-6219[5].
> >
> > Thanks,
> > Ji Liu
> >
> > [1] https://github.com/apache/arrow/pull/4960
> > [2] https://arrow.apache.org/docs/ipc.html
> > [3] https://issues.apache.org/jira/browse/ARROW-1875
> > [4] https://issues.apache.org/jira/browse/ARROW-6202[5] 
> > https://issues.apache.org/jira/browse/ARROW-6219
> >
> >
> >
> > --
> > From:Wes McKinney 
> > Send Time:2019年8月19日(星期一) 23:03
> > To:dev 
> > Subject:Re: Timeline for 0.15.0 release
> >
> > I'm going to work some on organizing the 0.15.0 backlog some this
> > week, if anyone wants to help with grooming (particularly for
> > languages other than C++/Python where I'm focusing) that would be
> > helpful. There have been almost 500 JIRA issues opened since the
> > 0.14.0 release, so we should make sure to check whether there's any
> > regressions or other serious bugs that we should try to fix for
> > 0.15.0.
> >
> > On Thu, Aug 15, 2019 at 6:23 PM Wes McKinney  wrote:
> > >
> > > The Windows wheel issue in 0.14.1 seems to be
> > >
> > > https://issues.apache.org/jira/browse/ARROW-6015
> > >
> > > I think the root cause could be the Windows changes in
> > >
> > > https://github.com/apache/arrow/commit/223ae744cc2a12c60cecb5db593263a03c13f85a
> > >
> > > I would be appreciative if a volunteer would look into what was wrong
> > > with the 0.14.1 wheels on Windows. Otherwise 0.15.0 Windows wheels
> > > will be broken, too
> > >
> > > The bad wheels can be found at
> > >
> > > https://bintray.com/apache/arrow/python#files/python%2F0.14.1
> > >
> > > On Thu, Aug 15, 2019 at 1:28 PM Antoine Pitrou  
> > > wrote:
> > > >
> > > > On Thu, 15 Aug 2019 11:17:07 -0700
> > > > Micah Kornfield  wrote:
> > > > > >
> > > > > > In C++ they are
> > > > > > independent, we could have 32-bit array lengths and variable-length
> > > > > > types with 64-bit offsets if we wanted (we just wouldn't be able to
> > > > > > have a List child with more than INT32_MAX elements).
> > > > >
> > > > > I think the point is we could do this in C++ but we don't.  I'm not 
> > > > > sure we
> > > > > would have introduced the "Large" types if we did.
> > > >
> > > > 64-bit offsets take twice as much space as 32-bit offsets, so if you're
> > > > storing lots of small-ish lists or strings, 32-bit offsets are
> > > > preferrable.  So even with 64-bit array lengths from the start it would
> > > > still be beneficial to have types with 32-bit offsets.
> > > >
> > > > > Going with the limited address space in Java and calling it a 
> > > > > reference
> > > > > implementation seems suboptimal. If a consumer uses a "Large" type
> > > > > presumably it is because they need the ability to store more than 
> > > > > INT32_MAX
> > > > > child elements in a column, otherwise it is just wasting space [1].
> > > >
> > > > Probably. Though if the individual elements (lists or strings) are
> > > > large, not much space is wasted in proportion, so it may be simpler in
> > > > such a 

Re: Timeline for 0.15.0 release

2019-08-27 Thread Wes McKinney
hi,

I think we should try to release the week of September 9, so
development work should be completed by end of next week.

Does that seem reasonable?

I plan to get up a patch for the protocol alignment changes for C++ in
the next couple of days -- I think that getting the alignment work
done is the main barrier to releasing.

Thanks
Wes

On Mon, Aug 19, 2019 at 12:25 PM Ji Liu  wrote:
>
> Hi, Wes, on the java side, I can think of several bugs that need to be fixed 
> or reminded.
>
> i. ARROW-6040: Dictionary entries are required in IPC streams even when 
> empty[1]
> This one is under review now, however through this PR we find that there 
> seems a bug in java reading and writing dictionaries in IPC which is 
> Inconsistent with spec[2] since it assumes all dictionaries are at the start 
> of stream (see details in PR comments,  and this fix may not catch up with 
> version 0.15). @Micah Kornfield
>
> ii. ARROW-1875: Write 64-bit ints as strings in integration test JSON files[3]
> Java side code already checked in, other implementations seems not.
>
> iii. ARROW-6202: OutOfMemory in JdbcAdapter[4]
> Caused by trying to load all records in one contiguous batch, fixed by 
> providing iterator API for iteratively reading in ARROW-6219[5].
>
> Thanks,
> Ji Liu
>
> [1] https://github.com/apache/arrow/pull/4960
> [2] https://arrow.apache.org/docs/ipc.html
> [3] https://issues.apache.org/jira/browse/ARROW-1875
> [4] https://issues.apache.org/jira/browse/ARROW-6202[5] 
> https://issues.apache.org/jira/browse/ARROW-6219
>
>
>
> --
> From:Wes McKinney 
> Send Time:2019年8月19日(星期一) 23:03
> To:dev 
> Subject:Re: Timeline for 0.15.0 release
>
> I'm going to work some on organizing the 0.15.0 backlog some this
> week, if anyone wants to help with grooming (particularly for
> languages other than C++/Python where I'm focusing) that would be
> helpful. There have been almost 500 JIRA issues opened since the
> 0.14.0 release, so we should make sure to check whether there's any
> regressions or other serious bugs that we should try to fix for
> 0.15.0.
>
> On Thu, Aug 15, 2019 at 6:23 PM Wes McKinney  wrote:
> >
> > The Windows wheel issue in 0.14.1 seems to be
> >
> > https://issues.apache.org/jira/browse/ARROW-6015
> >
> > I think the root cause could be the Windows changes in
> >
> > https://github.com/apache/arrow/commit/223ae744cc2a12c60cecb5db593263a03c13f85a
> >
> > I would be appreciative if a volunteer would look into what was wrong
> > with the 0.14.1 wheels on Windows. Otherwise 0.15.0 Windows wheels
> > will be broken, too
> >
> > The bad wheels can be found at
> >
> > https://bintray.com/apache/arrow/python#files/python%2F0.14.1
> >
> > On Thu, Aug 15, 2019 at 1:28 PM Antoine Pitrou  wrote:
> > >
> > > On Thu, 15 Aug 2019 11:17:07 -0700
> > > Micah Kornfield  wrote:
> > > > >
> > > > > In C++ they are
> > > > > independent, we could have 32-bit array lengths and variable-length
> > > > > types with 64-bit offsets if we wanted (we just wouldn't be able to
> > > > > have a List child with more than INT32_MAX elements).
> > > >
> > > > I think the point is we could do this in C++ but we don't.  I'm not 
> > > > sure we
> > > > would have introduced the "Large" types if we did.
> > >
> > > 64-bit offsets take twice as much space as 32-bit offsets, so if you're
> > > storing lots of small-ish lists or strings, 32-bit offsets are
> > > preferrable.  So even with 64-bit array lengths from the start it would
> > > still be beneficial to have types with 32-bit offsets.
> > >
> > > > Going with the limited address space in Java and calling it a reference
> > > > implementation seems suboptimal. If a consumer uses a "Large" type
> > > > presumably it is because they need the ability to store more than 
> > > > INT32_MAX
> > > > child elements in a column, otherwise it is just wasting space [1].
> > >
> > > Probably. Though if the individual elements (lists or strings) are
> > > large, not much space is wasted in proportion, so it may be simpler in
> > > such a case to always create a "Large" type array.
> > >
> > > > [1] I suppose theoretically there might be some performance benefits on
> > > > 64-bit architectures to using the native word sizes.
> > >
> > > Concretely, common 64-bit architectures don't do that, as 32-bit is an
> > > extremely common integer size even in high-performance code.
> > >
> > > Regards
> > >
> > > Antoine.
> > >
> > >
>


Re: Timeline for 0.15.0 release

2019-08-19 Thread Ji Liu
Hi, Wes, on the java side, I can think of several bugs that need to be fixed or 
reminded.

i. ARROW-6040: Dictionary entries are required in IPC streams even when empty[1]
This one is under review now, however through this PR we find that there seems 
a bug in java reading and writing dictionaries in IPC which is Inconsistent 
with spec[2] since it assumes all dictionaries are at the start of stream (see 
details in PR comments,  and this fix may not catch up with version 0.15). 
@Micah Kornfield

ii. ARROW-1875: Write 64-bit ints as strings in integration test JSON files[3]
Java side code already checked in, other implementations seems not.

iii. ARROW-6202: OutOfMemory in JdbcAdapter[4]
Caused by trying to load all records in one contiguous batch, fixed by 
providing iterator API for iteratively reading in ARROW-6219[5].

Thanks,
Ji Liu

[1] https://github.com/apache/arrow/pull/4960
[2] https://arrow.apache.org/docs/ipc.html
[3] https://issues.apache.org/jira/browse/ARROW-1875
[4] https://issues.apache.org/jira/browse/ARROW-6202[5] 
https://issues.apache.org/jira/browse/ARROW-6219



--
From:Wes McKinney 
Send Time:2019年8月19日(星期一) 23:03
To:dev 
Subject:Re: Timeline for 0.15.0 release

I'm going to work some on organizing the 0.15.0 backlog some this
week, if anyone wants to help with grooming (particularly for
languages other than C++/Python where I'm focusing) that would be
helpful. There have been almost 500 JIRA issues opened since the
0.14.0 release, so we should make sure to check whether there's any
regressions or other serious bugs that we should try to fix for
0.15.0.

On Thu, Aug 15, 2019 at 6:23 PM Wes McKinney  wrote:
>
> The Windows wheel issue in 0.14.1 seems to be
>
> https://issues.apache.org/jira/browse/ARROW-6015
>
> I think the root cause could be the Windows changes in
>
> https://github.com/apache/arrow/commit/223ae744cc2a12c60cecb5db593263a03c13f85a
>
> I would be appreciative if a volunteer would look into what was wrong
> with the 0.14.1 wheels on Windows. Otherwise 0.15.0 Windows wheels
> will be broken, too
>
> The bad wheels can be found at
>
> https://bintray.com/apache/arrow/python#files/python%2F0.14.1
>
> On Thu, Aug 15, 2019 at 1:28 PM Antoine Pitrou  wrote:
> >
> > On Thu, 15 Aug 2019 11:17:07 -0700
> > Micah Kornfield  wrote:
> > > >
> > > > In C++ they are
> > > > independent, we could have 32-bit array lengths and variable-length
> > > > types with 64-bit offsets if we wanted (we just wouldn't be able to
> > > > have a List child with more than INT32_MAX elements).
> > >
> > > I think the point is we could do this in C++ but we don't.  I'm not sure 
> > > we
> > > would have introduced the "Large" types if we did.
> >
> > 64-bit offsets take twice as much space as 32-bit offsets, so if you're
> > storing lots of small-ish lists or strings, 32-bit offsets are
> > preferrable.  So even with 64-bit array lengths from the start it would
> > still be beneficial to have types with 32-bit offsets.
> >
> > > Going with the limited address space in Java and calling it a reference
> > > implementation seems suboptimal. If a consumer uses a "Large" type
> > > presumably it is because they need the ability to store more than 
> > > INT32_MAX
> > > child elements in a column, otherwise it is just wasting space [1].
> >
> > Probably. Though if the individual elements (lists or strings) are
> > large, not much space is wasted in proportion, so it may be simpler in
> > such a case to always create a "Large" type array.
> >
> > > [1] I suppose theoretically there might be some performance benefits on
> > > 64-bit architectures to using the native word sizes.
> >
> > Concretely, common 64-bit architectures don't do that, as 32-bit is an
> > extremely common integer size even in high-performance code.
> >
> > Regards
> >
> > Antoine.
> >
> >



Re: Timeline for 0.15.0 release

2019-08-19 Thread Wes McKinney
I'm going to work some on organizing the 0.15.0 backlog some this
week, if anyone wants to help with grooming (particularly for
languages other than C++/Python where I'm focusing) that would be
helpful. There have been almost 500 JIRA issues opened since the
0.14.0 release, so we should make sure to check whether there's any
regressions or other serious bugs that we should try to fix for
0.15.0.

On Thu, Aug 15, 2019 at 6:23 PM Wes McKinney  wrote:
>
> The Windows wheel issue in 0.14.1 seems to be
>
> https://issues.apache.org/jira/browse/ARROW-6015
>
> I think the root cause could be the Windows changes in
>
> https://github.com/apache/arrow/commit/223ae744cc2a12c60cecb5db593263a03c13f85a
>
> I would be appreciative if a volunteer would look into what was wrong
> with the 0.14.1 wheels on Windows. Otherwise 0.15.0 Windows wheels
> will be broken, too
>
> The bad wheels can be found at
>
> https://bintray.com/apache/arrow/python#files/python%2F0.14.1
>
> On Thu, Aug 15, 2019 at 1:28 PM Antoine Pitrou  wrote:
> >
> > On Thu, 15 Aug 2019 11:17:07 -0700
> > Micah Kornfield  wrote:
> > > >
> > > > In C++ they are
> > > > independent, we could have 32-bit array lengths and variable-length
> > > > types with 64-bit offsets if we wanted (we just wouldn't be able to
> > > > have a List child with more than INT32_MAX elements).
> > >
> > > I think the point is we could do this in C++ but we don't.  I'm not sure 
> > > we
> > > would have introduced the "Large" types if we did.
> >
> > 64-bit offsets take twice as much space as 32-bit offsets, so if you're
> > storing lots of small-ish lists or strings, 32-bit offsets are
> > preferrable.  So even with 64-bit array lengths from the start it would
> > still be beneficial to have types with 32-bit offsets.
> >
> > > Going with the limited address space in Java and calling it a reference
> > > implementation seems suboptimal. If a consumer uses a "Large" type
> > > presumably it is because they need the ability to store more than 
> > > INT32_MAX
> > > child elements in a column, otherwise it is just wasting space [1].
> >
> > Probably. Though if the individual elements (lists or strings) are
> > large, not much space is wasted in proportion, so it may be simpler in
> > such a case to always create a "Large" type array.
> >
> > > [1] I suppose theoretically there might be some performance benefits on
> > > 64-bit architectures to using the native word sizes.
> >
> > Concretely, common 64-bit architectures don't do that, as 32-bit is an
> > extremely common integer size even in high-performance code.
> >
> > Regards
> >
> > Antoine.
> >
> >


Re: Timeline for 0.15.0 release

2019-08-15 Thread Wes McKinney
The Windows wheel issue in 0.14.1 seems to be

https://issues.apache.org/jira/browse/ARROW-6015

I think the root cause could be the Windows changes in

https://github.com/apache/arrow/commit/223ae744cc2a12c60cecb5db593263a03c13f85a

I would be appreciative if a volunteer would look into what was wrong
with the 0.14.1 wheels on Windows. Otherwise 0.15.0 Windows wheels
will be broken, too

The bad wheels can be found at

https://bintray.com/apache/arrow/python#files/python%2F0.14.1

On Thu, Aug 15, 2019 at 1:28 PM Antoine Pitrou  wrote:
>
> On Thu, 15 Aug 2019 11:17:07 -0700
> Micah Kornfield  wrote:
> > >
> > > In C++ they are
> > > independent, we could have 32-bit array lengths and variable-length
> > > types with 64-bit offsets if we wanted (we just wouldn't be able to
> > > have a List child with more than INT32_MAX elements).
> >
> > I think the point is we could do this in C++ but we don't.  I'm not sure we
> > would have introduced the "Large" types if we did.
>
> 64-bit offsets take twice as much space as 32-bit offsets, so if you're
> storing lots of small-ish lists or strings, 32-bit offsets are
> preferrable.  So even with 64-bit array lengths from the start it would
> still be beneficial to have types with 32-bit offsets.
>
> > Going with the limited address space in Java and calling it a reference
> > implementation seems suboptimal. If a consumer uses a "Large" type
> > presumably it is because they need the ability to store more than INT32_MAX
> > child elements in a column, otherwise it is just wasting space [1].
>
> Probably. Though if the individual elements (lists or strings) are
> large, not much space is wasted in proportion, so it may be simpler in
> such a case to always create a "Large" type array.
>
> > [1] I suppose theoretically there might be some performance benefits on
> > 64-bit architectures to using the native word sizes.
>
> Concretely, common 64-bit architectures don't do that, as 32-bit is an
> extremely common integer size even in high-performance code.
>
> Regards
>
> Antoine.
>
>


Re: Timeline for 0.15.0 release

2019-08-15 Thread Antoine Pitrou
On Thu, 15 Aug 2019 11:17:07 -0700
Micah Kornfield  wrote:
> >
> > In C++ they are
> > independent, we could have 32-bit array lengths and variable-length
> > types with 64-bit offsets if we wanted (we just wouldn't be able to
> > have a List child with more than INT32_MAX elements).  
> 
> I think the point is we could do this in C++ but we don't.  I'm not sure we
> would have introduced the "Large" types if we did.

64-bit offsets take twice as much space as 32-bit offsets, so if you're
storing lots of small-ish lists or strings, 32-bit offsets are
preferrable.  So even with 64-bit array lengths from the start it would
still be beneficial to have types with 32-bit offsets.

> Going with the limited address space in Java and calling it a reference
> implementation seems suboptimal. If a consumer uses a "Large" type
> presumably it is because they need the ability to store more than INT32_MAX
> child elements in a column, otherwise it is just wasting space [1].

Probably. Though if the individual elements (lists or strings) are
large, not much space is wasted in proportion, so it may be simpler in
such a case to always create a "Large" type array.

> [1] I suppose theoretically there might be some performance benefits on
> 64-bit architectures to using the native word sizes.

Concretely, common 64-bit architectures don't do that, as 32-bit is an
extremely common integer size even in high-performance code.

Regards

Antoine.




Re: Timeline for 0.15.0 release

2019-08-15 Thread Micah Kornfield
>
> In C++ they are
> independent, we could have 32-bit array lengths and variable-length
> types with 64-bit offsets if we wanted (we just wouldn't be able to
> have a List child with more than INT32_MAX elements).

I think the point is we could do this in C++ but we don't.  I'm not sure we
would have introduced the "Large" types if we did.
We will have to do this Java, it we don't want to convert to 64-bit
addressing.

Going with the limited address space in Java and calling it a reference
implementation seems suboptimal. If a consumer uses a "Large" type
presumably it is because they need the ability to store more than INT32_MAX
child elements in a column, otherwise it is just wasting space [1].

Let's pause until next week when Jacques is back online (and continue on
the other thread).  Like I said I think there is enough time either way to
get something in along the timeline we expect for the next release.

[1] I suppose theoretically there might be some performance benefits on
64-bit architectures to using the native word sizes.

On Thu, Aug 15, 2019 at 10:59 AM Wes McKinney  wrote:

> On Thu, Aug 15, 2019 at 12:00 AM Micah Kornfield 
> wrote:
> >
> > Hi Wes,
> > >
> > > Do these need to be dependent on the 64-bit array length discussion?
> >
> > We could hack something that can read the lower 32-bit range, so I guess
> > not, but this leaves a bad taste in my mouth.  I think there is likely
> > still enough time to have the discussion and get these implemented, one
> way
> > or another.
> >
>
> I guess I still don't understand how the array lengths and the
> List/Varchar offsets are related to each other. I probably just
> haven't looked at the Java library enough. In C++ they are
> independent, we could have 32-bit array lengths and variable-length
> types with 64-bit offsets if we wanted (we just wouldn't be able to
> have a List child with more than INT32_MAX elements). We would have to
> do a limited amount of boundschecking at IPC boundary points (like
> Java is checking presumably now for vectors exceeding INT32_MAX).
>
> > For the record, I don't think we should hold a major release hostage
> > > if we aren't able to complete various feature milestones in time.
> > > Since it's been about 5-6 weeks since 0.14.0 we're coming close to the
> > > desired 8-10 week timeline for major releases, so if we need to have
> > > 0.16.0 prior to 1.0.0, I think that is OK also.
> >
> > I agree with the time based milestones in practice, but we are
> backpedaling
> > on the intent to keep type parity between the two reference
> > implementations.  At least the way I read the previous threads on the
> > topic, I thought there was lazy consensus that in lieu of requiring
> working
> > implementations in Java and C++ be checked in at the same time, we would
> > rely on the release as a mechanism to forcing function for parity.
> >
>
> I agree with the intent and spirit of the idea, but it seems we have a
> can of worms on our hands now and so I don't think we should keep from
> releasing the work that has been completed if consensus about Java
> changes is not reached in time.
>
> > Thanks,
> > Micah
> >
> > On Wed, Aug 14, 2019 at 11:32 AM Antoine Pitrou 
> wrote:
> >
> > >
> > > Agreed with Wes.
> > >
> > > Regards
> > >
> > > Antoine.
> > >
> > >
> > > Le 14/08/2019 à 20:30, Wes McKinney a écrit :
> > > > For the record, I don't think we should hold a major release hostage
> > > > if we aren't able to complete various feature milestones in time.
> > > > Since it's been about 5-6 weeks since 0.14.0 we're coming close to
> the
> > > > desired 8-10 week timeline for major releases, so if we need to have
> > > > 0.16.0 prior to 1.0.0, I think that is OK also.
> > > >
> > > > On Wed, Aug 14, 2019 at 11:45 AM Wes McKinney 
> > > wrote:
> > > >>
> > > >> On Wed, Aug 14, 2019 at 11:43 AM Micah Kornfield <
> emkornfi...@gmail.com>
> > > wrote:
> > > >>>
> > > 
> > >   is there anything else that has come up that
> > >  definitely needs to happen before we can release again?
> > > >>>
> > > >>> We need to decide on a way forward for LargeList, LargeBinary, etc,
> > > types...
> > > >>>
> > > >>
> > > >> Do these need to be dependent on the 64-bit array length discussion?
> > > >> They seem somewhat orthogonal to me. If we have to release 0.15.0
> > > >> without the Java side of these, that's OK with me, since reaching
> > > >> format implementation completeness is more of a 1.0.0 concern
> > > >>
> > > >>> On Tue, Aug 13, 2019 at 8:27 PM Wes McKinney 
> > > wrote:
> > > >>>
> > >  hi folks,
> > > 
> > >  Since there have been a number of fairly serious issues (e.g.
> > >  ARROW-6060) since 0.14.1 that have been fixed I think we should
> start
> > >  planning of the next major release. Note that we still have some
> > >  format-related work (the Flatbuffers alignment issue) that ought
> to be
> > >  resolved (not a small task since it affects 4 or 5
> implementations),
> > > 

Re: Timeline for 0.15.0 release

2019-08-15 Thread Wes McKinney
On Thu, Aug 15, 2019 at 12:00 AM Micah Kornfield  wrote:
>
> Hi Wes,
> >
> > Do these need to be dependent on the 64-bit array length discussion?
>
> We could hack something that can read the lower 32-bit range, so I guess
> not, but this leaves a bad taste in my mouth.  I think there is likely
> still enough time to have the discussion and get these implemented, one way
> or another.
>

I guess I still don't understand how the array lengths and the
List/Varchar offsets are related to each other. I probably just
haven't looked at the Java library enough. In C++ they are
independent, we could have 32-bit array lengths and variable-length
types with 64-bit offsets if we wanted (we just wouldn't be able to
have a List child with more than INT32_MAX elements). We would have to
do a limited amount of boundschecking at IPC boundary points (like
Java is checking presumably now for vectors exceeding INT32_MAX).

> For the record, I don't think we should hold a major release hostage
> > if we aren't able to complete various feature milestones in time.
> > Since it's been about 5-6 weeks since 0.14.0 we're coming close to the
> > desired 8-10 week timeline for major releases, so if we need to have
> > 0.16.0 prior to 1.0.0, I think that is OK also.
>
> I agree with the time based milestones in practice, but we are backpedaling
> on the intent to keep type parity between the two reference
> implementations.  At least the way I read the previous threads on the
> topic, I thought there was lazy consensus that in lieu of requiring working
> implementations in Java and C++ be checked in at the same time, we would
> rely on the release as a mechanism to forcing function for parity.
>

I agree with the intent and spirit of the idea, but it seems we have a
can of worms on our hands now and so I don't think we should keep from
releasing the work that has been completed if consensus about Java
changes is not reached in time.

> Thanks,
> Micah
>
> On Wed, Aug 14, 2019 at 11:32 AM Antoine Pitrou  wrote:
>
> >
> > Agreed with Wes.
> >
> > Regards
> >
> > Antoine.
> >
> >
> > Le 14/08/2019 à 20:30, Wes McKinney a écrit :
> > > For the record, I don't think we should hold a major release hostage
> > > if we aren't able to complete various feature milestones in time.
> > > Since it's been about 5-6 weeks since 0.14.0 we're coming close to the
> > > desired 8-10 week timeline for major releases, so if we need to have
> > > 0.16.0 prior to 1.0.0, I think that is OK also.
> > >
> > > On Wed, Aug 14, 2019 at 11:45 AM Wes McKinney 
> > wrote:
> > >>
> > >> On Wed, Aug 14, 2019 at 11:43 AM Micah Kornfield 
> > wrote:
> > >>>
> > 
> >   is there anything else that has come up that
> >  definitely needs to happen before we can release again?
> > >>>
> > >>> We need to decide on a way forward for LargeList, LargeBinary, etc,
> > types...
> > >>>
> > >>
> > >> Do these need to be dependent on the 64-bit array length discussion?
> > >> They seem somewhat orthogonal to me. If we have to release 0.15.0
> > >> without the Java side of these, that's OK with me, since reaching
> > >> format implementation completeness is more of a 1.0.0 concern
> > >>
> > >>> On Tue, Aug 13, 2019 at 8:27 PM Wes McKinney 
> > wrote:
> > >>>
> >  hi folks,
> > 
> >  Since there have been a number of fairly serious issues (e.g.
> >  ARROW-6060) since 0.14.1 that have been fixed I think we should start
> >  planning of the next major release. Note that we still have some
> >  format-related work (the Flatbuffers alignment issue) that ought to be
> >  resolved (not a small task since it affects 4 or 5 implementations),
> >  but aside from that, is there anything else that has come up that
> >  definitely needs to happen before we can release again?
> > 
> >  I would say cutting a release somewhere around the US Labor Day
> >  holiday (~the week after or so) would be called for.
> > 
> >  Thanks,
> >  Wes
> > 
> >


Re: Timeline for 0.15.0 release

2019-08-14 Thread Micah Kornfield
Hi Wes,
>
> Do these need to be dependent on the 64-bit array length discussion?

We could hack something that can read the lower 32-bit range, so I guess
not, but this leaves a bad taste in my mouth.  I think there is likely
still enough time to have the discussion and get these implemented, one way
or another.

For the record, I don't think we should hold a major release hostage
> if we aren't able to complete various feature milestones in time.
> Since it's been about 5-6 weeks since 0.14.0 we're coming close to the
> desired 8-10 week timeline for major releases, so if we need to have
> 0.16.0 prior to 1.0.0, I think that is OK also.

I agree with the time based milestones in practice, but we are backpedaling
on the intent to keep type parity between the two reference
implementations.  At least the way I read the previous threads on the
topic, I thought there was lazy consensus that in lieu of requiring working
implementations in Java and C++ be checked in at the same time, we would
rely on the release as a mechanism to forcing function for parity.

Thanks,
Micah

On Wed, Aug 14, 2019 at 11:32 AM Antoine Pitrou  wrote:

>
> Agreed with Wes.
>
> Regards
>
> Antoine.
>
>
> Le 14/08/2019 à 20:30, Wes McKinney a écrit :
> > For the record, I don't think we should hold a major release hostage
> > if we aren't able to complete various feature milestones in time.
> > Since it's been about 5-6 weeks since 0.14.0 we're coming close to the
> > desired 8-10 week timeline for major releases, so if we need to have
> > 0.16.0 prior to 1.0.0, I think that is OK also.
> >
> > On Wed, Aug 14, 2019 at 11:45 AM Wes McKinney 
> wrote:
> >>
> >> On Wed, Aug 14, 2019 at 11:43 AM Micah Kornfield 
> wrote:
> >>>
> 
>   is there anything else that has come up that
>  definitely needs to happen before we can release again?
> >>>
> >>> We need to decide on a way forward for LargeList, LargeBinary, etc,
> types...
> >>>
> >>
> >> Do these need to be dependent on the 64-bit array length discussion?
> >> They seem somewhat orthogonal to me. If we have to release 0.15.0
> >> without the Java side of these, that's OK with me, since reaching
> >> format implementation completeness is more of a 1.0.0 concern
> >>
> >>> On Tue, Aug 13, 2019 at 8:27 PM Wes McKinney 
> wrote:
> >>>
>  hi folks,
> 
>  Since there have been a number of fairly serious issues (e.g.
>  ARROW-6060) since 0.14.1 that have been fixed I think we should start
>  planning of the next major release. Note that we still have some
>  format-related work (the Flatbuffers alignment issue) that ought to be
>  resolved (not a small task since it affects 4 or 5 implementations),
>  but aside from that, is there anything else that has come up that
>  definitely needs to happen before we can release again?
> 
>  I would say cutting a release somewhere around the US Labor Day
>  holiday (~the week after or so) would be called for.
> 
>  Thanks,
>  Wes
> 
>


Re: Timeline for 0.15.0 release

2019-08-14 Thread Antoine Pitrou


Agreed with Wes.

Regards

Antoine.


Le 14/08/2019 à 20:30, Wes McKinney a écrit :
> For the record, I don't think we should hold a major release hostage
> if we aren't able to complete various feature milestones in time.
> Since it's been about 5-6 weeks since 0.14.0 we're coming close to the
> desired 8-10 week timeline for major releases, so if we need to have
> 0.16.0 prior to 1.0.0, I think that is OK also.
> 
> On Wed, Aug 14, 2019 at 11:45 AM Wes McKinney  wrote:
>>
>> On Wed, Aug 14, 2019 at 11:43 AM Micah Kornfield  
>> wrote:
>>>

  is there anything else that has come up that
 definitely needs to happen before we can release again?
>>>
>>> We need to decide on a way forward for LargeList, LargeBinary, etc, types...
>>>
>>
>> Do these need to be dependent on the 64-bit array length discussion?
>> They seem somewhat orthogonal to me. If we have to release 0.15.0
>> without the Java side of these, that's OK with me, since reaching
>> format implementation completeness is more of a 1.0.0 concern
>>
>>> On Tue, Aug 13, 2019 at 8:27 PM Wes McKinney  wrote:
>>>
 hi folks,

 Since there have been a number of fairly serious issues (e.g.
 ARROW-6060) since 0.14.1 that have been fixed I think we should start
 planning of the next major release. Note that we still have some
 format-related work (the Flatbuffers alignment issue) that ought to be
 resolved (not a small task since it affects 4 or 5 implementations),
 but aside from that, is there anything else that has come up that
 definitely needs to happen before we can release again?

 I would say cutting a release somewhere around the US Labor Day
 holiday (~the week after or so) would be called for.

 Thanks,
 Wes



Re: Timeline for 0.15.0 release

2019-08-14 Thread Wes McKinney
For the record, I don't think we should hold a major release hostage
if we aren't able to complete various feature milestones in time.
Since it's been about 5-6 weeks since 0.14.0 we're coming close to the
desired 8-10 week timeline for major releases, so if we need to have
0.16.0 prior to 1.0.0, I think that is OK also.

On Wed, Aug 14, 2019 at 11:45 AM Wes McKinney  wrote:
>
> On Wed, Aug 14, 2019 at 11:43 AM Micah Kornfield  
> wrote:
> >
> > >
> > >  is there anything else that has come up that
> > > definitely needs to happen before we can release again?
> >
> > We need to decide on a way forward for LargeList, LargeBinary, etc, types...
> >
>
> Do these need to be dependent on the 64-bit array length discussion?
> They seem somewhat orthogonal to me. If we have to release 0.15.0
> without the Java side of these, that's OK with me, since reaching
> format implementation completeness is more of a 1.0.0 concern
>
> > On Tue, Aug 13, 2019 at 8:27 PM Wes McKinney  wrote:
> >
> > > hi folks,
> > >
> > > Since there have been a number of fairly serious issues (e.g.
> > > ARROW-6060) since 0.14.1 that have been fixed I think we should start
> > > planning of the next major release. Note that we still have some
> > > format-related work (the Flatbuffers alignment issue) that ought to be
> > > resolved (not a small task since it affects 4 or 5 implementations),
> > > but aside from that, is there anything else that has come up that
> > > definitely needs to happen before we can release again?
> > >
> > > I would say cutting a release somewhere around the US Labor Day
> > > holiday (~the week after or so) would be called for.
> > >
> > > Thanks,
> > > Wes
> > >


Re: Timeline for 0.15.0 release

2019-08-14 Thread Wes McKinney
On Wed, Aug 14, 2019 at 11:43 AM Micah Kornfield  wrote:
>
> >
> >  is there anything else that has come up that
> > definitely needs to happen before we can release again?
>
> We need to decide on a way forward for LargeList, LargeBinary, etc, types...
>

Do these need to be dependent on the 64-bit array length discussion?
They seem somewhat orthogonal to me. If we have to release 0.15.0
without the Java side of these, that's OK with me, since reaching
format implementation completeness is more of a 1.0.0 concern

> On Tue, Aug 13, 2019 at 8:27 PM Wes McKinney  wrote:
>
> > hi folks,
> >
> > Since there have been a number of fairly serious issues (e.g.
> > ARROW-6060) since 0.14.1 that have been fixed I think we should start
> > planning of the next major release. Note that we still have some
> > format-related work (the Flatbuffers alignment issue) that ought to be
> > resolved (not a small task since it affects 4 or 5 implementations),
> > but aside from that, is there anything else that has come up that
> > definitely needs to happen before we can release again?
> >
> > I would say cutting a release somewhere around the US Labor Day
> > holiday (~the week after or so) would be called for.
> >
> > Thanks,
> > Wes
> >


Re: Timeline for 0.15.0 release

2019-08-14 Thread Micah Kornfield
>
>  is there anything else that has come up that
> definitely needs to happen before we can release again?

We need to decide on a way forward for LargeList, LargeBinary, etc, types...

On Tue, Aug 13, 2019 at 8:27 PM Wes McKinney  wrote:

> hi folks,
>
> Since there have been a number of fairly serious issues (e.g.
> ARROW-6060) since 0.14.1 that have been fixed I think we should start
> planning of the next major release. Note that we still have some
> format-related work (the Flatbuffers alignment issue) that ought to be
> resolved (not a small task since it affects 4 or 5 implementations),
> but aside from that, is there anything else that has come up that
> definitely needs to happen before we can release again?
>
> I would say cutting a release somewhere around the US Labor Day
> holiday (~the week after or so) would be called for.
>
> Thanks,
> Wes
>


Re: Timeline for 0.15.0 release

2019-08-14 Thread Wes McKinney
Is there a JIRA for the issue that caused us to pull the 0.14.1
Windows Python wheel installers? If we want to have working wheels for
0.15.0 we'll need a volunteer to help address whatever was wrong with
0.14.1.

On Tue, Aug 13, 2019 at 10:26 PM Wes McKinney  wrote:
>
> hi folks,
>
> Since there have been a number of fairly serious issues (e.g.
> ARROW-6060) since 0.14.1 that have been fixed I think we should start
> planning of the next major release. Note that we still have some
> format-related work (the Flatbuffers alignment issue) that ought to be
> resolved (not a small task since it affects 4 or 5 implementations),
> but aside from that, is there anything else that has come up that
> definitely needs to happen before we can release again?
>
> I would say cutting a release somewhere around the US Labor Day
> holiday (~the week after or so) would be called for.
>
> Thanks,
> Wes