Thanks, I'm trying to reproduce it now! On Thu, Mar 15, 2018 at 1:28 PM, Wes McKinney <wesmck...@gmail.com> wrote:
> Unfortunately, we aren't quite out of the woods yet. > > https://issues.apache.org/jira/browse/ARROW-2318 > > Robert or Philipp, could you please advise on this issue? > > I think if we're learning anything from this experience, we need to > work on our continuous integration and delivery. We shouldn't be > finding out about all these problems at the least minute -- the master > branch should be kept in a releasable state every day (i.e. the > release verification script passes on a snapshot of master) > > On Thu, Mar 15, 2018 at 2:06 AM, Kouhei Sutou <k...@clear-code.com> wrote: > > https://github.com/apache/arrow/pull/1752 will solve this. > > > > -- > > kou > > > > In <20180315.135146.1891617944364686374....@clear-code.com> > > "Re: Working towards getting 0.9.0 release candidate up next week" on > Thu, 15 Mar 2018 13:51:46 +0900 (JST), > > Kouhei Sutou <k...@clear-code.com> wrote: > > > >> Sure. Please wait for a while. > >> > >> -- > >> kou > >> > >> In <CAJPUwMCHqVSbJp5xVDXZV2qY=n5DVPqH=miaetgq_d8c5on...@mail.gmail.com> > >> "Re: Working towards getting 0.9.0 release candidate up next week" on > Thu, 15 Mar 2018 00:49:02 -0400, > >> Wes McKinney <wesmck...@gmail.com> wrote: > >> > >>> I ran into another problem: > >>> > >>> https://issues.apache.org/jira/browse/ARROW-2313 > >>> > >>> @Kou, could you have a look at this? Thanks > >>> > >>> On Wed, Mar 14, 2018 at 10:36 PM, Wes McKinney <wesmck...@gmail.com> > wrote: > >>>> Thanks Paul! This seems to have done the trick. I will get the RC out > >>>> later this evening > >>>> > >>>> On Wed, Mar 14, 2018 at 9:50 PM, Paul Taylor <ptay...@apache.org> > wrote: > >>>>> This should fix it: https://github.com/apache/arrow/pull/1751 < > https://github.com/apache/arrow/pull/1751> > >>>>> > >>>>>> On Mar 14, 2018, at 6:42 PM, Wes McKinney <wesmck...@gmail.com> > wrote: > >>>>>> > >>>>>> Last item https://issues.apache.org/jira/browse/ARROW-2312 > >>>>>> > >>>>>> I can start the release vote as soon as we have the release > >>>>>> verification script working again > >>>>>> > >>>>>> On Wed, Mar 14, 2018 at 2:28 PM, Wes McKinney <wesmck...@gmail.com> > wrote: > >>>>>>> OK, patch is up for ARROW-2307: > >>>>>>> https://github.com/apache/arrow/pull/1747. Once that is reviewed > and > >>>>>>> merged I will start the release vote > >>>>>>> > >>>>>>> On Wed, Mar 14, 2018 at 10:18 AM, Wes McKinney < > wesmck...@gmail.com> wrote: > >>>>>>>> I'm going to have a look at ARROW-2307 to see if it's an easy > fix. If > >>>>>>>> not, I will go ahead with the RC > >>>>>>>> > >>>>>>>> On Wed, Mar 14, 2018 at 7:36 AM, Uwe L. Korn <uw...@xhochy.com> > wrote: > >>>>>>>>> The mentioned bugfixes were merged and I also tested the > Arrow<->Parquet-cpp as well as the Arrow<->Dask(parquet) integration. Both > seem to work fine. From my side it looks like we're ready to make an RC. > >>>>>>>>> > >>>>>>>>> Uwe > >>>>>>>>> > >>>>>>>>> On Wed, Mar 14, 2018, at 5:07 AM, Wes McKinney wrote: > >>>>>>>>>> I fixed these bugs > >>>>>>>>>> > >>>>>>>>>> https://github.com/apache/arrow/pull/1742 > >>>>>>>>>> https://github.com/apache/arrow/pull/1743 > >>>>>>>>>> > >>>>>>>>>> As soon as these patches go in, we can cut the RC0. I can do > this as > >>>>>>>>>> soon as tomorrow (Wednesday) morning > >>>>>>>>>> > >>>>>>>>>> On Tue, Mar 13, 2018 at 5:00 PM, Wes McKinney < > wesmck...@gmail.com> wrote: > >>>>>>>>>>> I found 2 bugs -- ARROW-2304 and ARROW-2306 -- while doing > some final > >>>>>>>>>>> testing of master (stuff we haven't been testing in CI -- we > _really_ > >>>>>>>>>>> need to set up nightly CI jobs for more time consuming tests, > like > >>>>>>>>>>> HDFS, that we want to test periodically but perhaps not on > every > >>>>>>>>>>> commit). I'm going to see if these are easy to fix > >>>>>>>>>>> > >>>>>>>>>>> On Tue, Mar 13, 2018 at 11:21 AM, Wes McKinney < > wesmck...@gmail.com> wrote: > >>>>>>>>>>>> I fixed ARROW-2227 in https://github.com/apache/ > arrow/pull/1740, so if > >>>>>>>>>>>> someone could review that would be great. > >>>>>>>>>>>> > >>>>>>>>>>>> I am going to make sure the C++/Python HDFS tests run > locally, then > >>>>>>>>>>>> after the patch above is merged we should be good to make the > RC. > >>>>>>>>>>>> > >>>>>>>>>>>> Note to other onlookers -- please feel free to keep writing > new > >>>>>>>>>>>> patches not mentioned here. They may just have to go in to > the next > >>>>>>>>>>>> release > >>>>>>>>>>>> > >>>>>>>>>>>> Thanks > >>>>>>>>>>>> Wes > >>>>>>>>>>>> > >>>>>>>>>>>> On Tue, Mar 13, 2018 at 12:56 AM, Wes McKinney < > wesmck...@gmail.com> wrote: > >>>>>>>>>>>>> Things are looking pretty good. I'm waiting on a build for > ARROW-1643 > >>>>>>>>>>>>> to go in, and there's a last blocker bug (ARROW-2227) that > we ought to > >>>>>>>>>>>>> try to fix tomorrow before cutting the RC. I can start the > vote after > >>>>>>>>>>>>> that if nothing else comes up > >>>>>>>>>>>>> > >>>>>>>>>>>>> I created ARROW-2300 in the course of trying to test > ARROW-1643. I > >>>>>>>>>>>>> have an alternative way to run the HDFS tests (since we > aren't running > >>>>>>>>>>>>> them in Travis CI) so most likely I will try out the HDFS > tests and > >>>>>>>>>>>>> then move ARROW-2300 to the next release milestone. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thanks > >>>>>>>>>>>>> Wes > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Mon, Mar 12, 2018 at 12:03 AM, Wes McKinney < > wesmck...@gmail.com> wrote: > >>>>>>>>>>>>>> I've done a pass over the remaining JIRAs -- I think we're > going to > >>>>>>>>>>>>>> need at least another full day to get things buttoned up, > so I would > >>>>>>>>>>>>>> say we're looking at an RC on Tuesday. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> In progress: > >>>>>>>>>>>>>> - ARROW-1425: Patch up, needs some editing, may be moved to > 0.10.0 > >>>>>>>>>>>>>> - ARROW-2282: Patch up, needs some test cases > >>>>>>>>>>>>>> - ARROW-1974: Patch in review in apache/parquet-cpp -- may > want to > >>>>>>>>>>>>>> move this JIRA to the Parquet project > >>>>>>>>>>>>>> - ARROW-2122: Patch to be reviewed > >>>>>>>>>>>>>> - ARROW-2135: Patch to be merged > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> TODO > >>>>>>>>>>>>>> - ARROW-2082: Parquet segfault <- to be investigated, may > be moved to 0.10.0 > >>>>>>>>>>>>>> - ARROW-2118: Fix rough edge with reading length-0 files > >>>>>>>>>>>>>> - ARROW-2227: Bug with creating chunked arrays in > Table.from_pandas > >>>>>>>>>>>>>> - ARROW-2292: Deprecation / renaming a Python method > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Out of the two items in TODO, ARROW-2118 and ARROW-2292 are > small > >>>>>>>>>>>>>> matters, so I will take care of them. ARROW-2227 may not be > fixable > >>>>>>>>>>>>>> within ~1 day but it would be useful to have a diagnosis in > case the > >>>>>>>>>>>>>> fix is easy -- multiple users hit this bug. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>> Wes > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Thu, Mar 8, 2018 at 8:47 PM, Kouhei Sutou < > k...@clear-code.com> wrote: > >>>>>>>>>>>>>>> Thanks! > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>> kou > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> In <CAJPUwMBsxXW9-8VRJ3xWUd_P3Y7soSDMC6xc53PKrJuSeWmxdw@ > mail.gmail.com> > >>>>>>>>>>>>>>> "Re: Working towards getting 0.9.0 release candidate up > next week" on Thu, 8 Mar 2018 20:44:14 -0500, > >>>>>>>>>>>>>>> Wes McKinney <wesmck...@gmail.com> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> hi Kou -- yes, I think this is a good idea. It will > require a little > >>>>>>>>>>>>>>>> bit of work to be able to produce a viable standalone > source tarball. > >>>>>>>>>>>>>>>> Between Uwe, Phillip, Antoine, and I, we should be able > to come up > >>>>>>>>>>>>>>>> with a plan to do this > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> - Wes > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On Thu, Mar 8, 2018 at 8:33 PM, Kouhei Sutou < > k...@clear-code.com> wrote: > >>>>>>>>>>>>>>>>> Hi, > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> - Updating pip packages for C++ and Python > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Can we try adding PyArrow source package to PyPI at the > >>>>>>>>>>>>>>>>> 0.9.0? > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> I want to install PyArrow with Arrow C++ installed by > .deb > >>>>>>>>>>>>>>>>> or .rpm. I want to use both Red Arrow (Ruby bindings) and > >>>>>>>>>>>>>>>>> PyArrow in the same process via PyCall (Ruby library > >>>>>>>>>>>>>>>>> to integrate with Python). In the case, I need to use the > >>>>>>>>>>>>>>>>> same Arrow C++ in both Red Arrow and PyArrow. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Now, there are only binary packages for PyArrow at > >>>>>>>>>>>>>>>>> https://pypi.python.org/pypi/pyarrow . If there is a > source > >>>>>>>>>>>>>>>>> package for PyArrow at PyPI, I can install PyArrow with > >>>>>>>>>>>>>>>>> Arrow C++ installed by .deb or .rpm by "pip --no-binary > >>>>>>>>>>>>>>>>> pyarrow". > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Red Arrow can also use Arrow C++ installed by .deb or > .rpm. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>>>> kou > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> In <CAKHW1aqGv7SN70JewE5QmW00yeVX > _xg_zjinywgbo00+c4k...@mail.gmail.com> > >>>>>>>>>>>>>>>>> "Re: Working towards getting 0.9.0 release candidate up > next week" on Thu, 8 Mar 2018 11:25:32 -0800, > >>>>>>>>>>>>>>>>> Siddharth Teotia <siddha...@dremio.com> wrote: > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> All, > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> I plan to get RC out over the weekend or early Monday. > Is that fine with > >>>>>>>>>>>>>>>>>> everybody? > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> We have 6 items in progress -- > >>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/projects/ARROW/versions/ > 12341707#release-report-tab-body. > >>>>>>>>>>>>>>>>>> How do people feel about completing these JIRAs by > tomorrow? I am > >>>>>>>>>>>>>>>>>> completely fine with deferring the RC to early next > week (Mon/Tue/Wed) if > >>>>>>>>>>>>>>>>>> necessary. Just looking for consensus. Also, I suggest > that we defer the > >>>>>>>>>>>>>>>>>> ones with TODO status. I will do it later today unless > I hear otherwise. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> I was wondering if anyone else is interested in > collaborating for the > >>>>>>>>>>>>>>>>>> post-release tasks. As per > >>>>>>>>>>>>>>>>>> https://github.com/apache/ > arrow/blob/master/dev/release/RELEASE_MANAGEMENT.md, > >>>>>>>>>>>>>>>>>> following are the high level post-release tasks. Please > let me know if you > >>>>>>>>>>>>>>>>>> would like to take up something. I have written my name > against some of > >>>>>>>>>>>>>>>>>> them. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> - Updating the Arrow Website (Sidd) > >>>>>>>>>>>>>>>>>> - Uploading release artifacts to SVN -- looks like > PMC karma is needed > >>>>>>>>>>>>>>>>>> to do this > >>>>>>>>>>>>>>>>>> - Announcing release (Sidd) > >>>>>>>>>>>>>>>>>> - Updating website with new API documentation (Sidd) > >>>>>>>>>>>>>>>>>> - Updating pip packages for C++ and Python > >>>>>>>>>>>>>>>>>> - Updating conda packages for C++ and Python (Sidd) > >>>>>>>>>>>>>>>>>> - Updating Java Maven artifacts in Maven central > (Sidd) > >>>>>>>>>>>>>>>>>> - Release blog post > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> If anything is missing, please add to the above list. > It will be helpful > >>>>>>>>>>>>>>>>>> for tracking. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>>>> Sidd > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> On Sun, Mar 4, 2018 at 12:34 PM, Wes McKinney < > wesmck...@gmail.com> wrote: > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> hey Sidd, > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> The Python backlog is still in pretty rough shape. I'd > like to see if > >>>>>>>>>>>>>>>>>>> we can make an RC by Friday but if not we can defer to > Monday/Tuesday > >>>>>>>>>>>>>>>>>>> the following week (3/12 or 13). I will trim as much > as possible out > >>>>>>>>>>>>>>>>>>> of the current backlog to get things down to the > essential > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> - Wes > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> On Sun, Feb 25, 2018 at 11:58 AM, Siddharth Teotia < > siddha...@dremio.com> > >>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>> Sounds good. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Thanks > >>>>>>>>>>>>>>>>>>>> Sidd > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> On Feb 24, 2018 6:24 PM, "Wes McKinney" < > wesmck...@gmail.com> wrote: > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Hi Sidd, > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> I think we have too many bugs to make an RC this > coming week. I suggest > >>>>>>>>>>>>>>>>>>> we > >>>>>>>>>>>>>>>>>>>> defer to the following week. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Thanks > >>>>>>>>>>>>>>>>>>>> Wes > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> On Feb 24, 2018 7:09 PM, "Siddharth Teotia" < > siddha...@dremio.com> > >>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Hi All, > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> We currently have 10 issues in progress and PRs are > available for 8 of > >>>>>>>>>>>>>>>>>>>> them. In interest of getting a release candidate next > week, I would > >>>>>>>>>>>>>>>>>>> request > >>>>>>>>>>>>>>>>>>>> people to review PRs as soon as they can to help make > progress and close > >>>>>>>>>>>>>>>>>>>> out as many JIRAs as we can. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> There are 32 issues in TODO list and 25 of them are > not yet assigned. I > >>>>>>>>>>>>>>>>>>> am > >>>>>>>>>>>>>>>>>>>> planning to defer some of the unassigned ones later > today or tomorrow. It > >>>>>>>>>>>>>>>>>>>> would be good to soon grab/assign the issues that > people want to be fixed > >>>>>>>>>>>>>>>>>>>> for 0.9.0. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Here is the link to backlog: > >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/ > jira/projects/ARROW/versions/12341707 > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>>>>>> Sidd > >>>>>>>>>>>>>>>>>>> > >>>>> >
