Sure. Please wait for a while. -- kou
In <CAJPUwMCHqVSbJp5xVDXZV2qY=n5DVPqH=miaetgq_d8c5on...@mail.gmail.com> "Re: Working towards getting 0.9.0 release candidate up next week" on Thu, 15 Mar 2018 00:49:02 -0400, Wes McKinney <wesmck...@gmail.com> wrote: > I ran into another problem: > > https://issues.apache.org/jira/browse/ARROW-2313 > > @Kou, could you have a look at this? Thanks > > On Wed, Mar 14, 2018 at 10:36 PM, Wes McKinney <wesmck...@gmail.com> wrote: >> Thanks Paul! This seems to have done the trick. I will get the RC out >> later this evening >> >> On Wed, Mar 14, 2018 at 9:50 PM, Paul Taylor <ptay...@apache.org> wrote: >>> This should fix it: https://github.com/apache/arrow/pull/1751 >>> <https://github.com/apache/arrow/pull/1751> >>> >>>> On Mar 14, 2018, at 6:42 PM, Wes McKinney <wesmck...@gmail.com> wrote: >>>> >>>> Last item https://issues.apache.org/jira/browse/ARROW-2312 >>>> >>>> I can start the release vote as soon as we have the release >>>> verification script working again >>>> >>>> On Wed, Mar 14, 2018 at 2:28 PM, Wes McKinney <wesmck...@gmail.com> wrote: >>>>> OK, patch is up for ARROW-2307: >>>>> https://github.com/apache/arrow/pull/1747. Once that is reviewed and >>>>> merged I will start the release vote >>>>> >>>>> On Wed, Mar 14, 2018 at 10:18 AM, Wes McKinney <wesmck...@gmail.com> >>>>> wrote: >>>>>> I'm going to have a look at ARROW-2307 to see if it's an easy fix. If >>>>>> not, I will go ahead with the RC >>>>>> >>>>>> On Wed, Mar 14, 2018 at 7:36 AM, Uwe L. Korn <uw...@xhochy.com> wrote: >>>>>>> The mentioned bugfixes were merged and I also tested the >>>>>>> Arrow<->Parquet-cpp as well as the Arrow<->Dask(parquet) integration. >>>>>>> Both seem to work fine. From my side it looks like we're ready to make >>>>>>> an RC. >>>>>>> >>>>>>> Uwe >>>>>>> >>>>>>> On Wed, Mar 14, 2018, at 5:07 AM, Wes McKinney wrote: >>>>>>>> I fixed these bugs >>>>>>>> >>>>>>>> https://github.com/apache/arrow/pull/1742 >>>>>>>> https://github.com/apache/arrow/pull/1743 >>>>>>>> >>>>>>>> As soon as these patches go in, we can cut the RC0. I can do this as >>>>>>>> soon as tomorrow (Wednesday) morning >>>>>>>> >>>>>>>> On Tue, Mar 13, 2018 at 5:00 PM, Wes McKinney <wesmck...@gmail.com> >>>>>>>> wrote: >>>>>>>>> I found 2 bugs -- ARROW-2304 and ARROW-2306 -- while doing some final >>>>>>>>> testing of master (stuff we haven't been testing in CI -- we _really_ >>>>>>>>> need to set up nightly CI jobs for more time consuming tests, like >>>>>>>>> HDFS, that we want to test periodically but perhaps not on every >>>>>>>>> commit). I'm going to see if these are easy to fix >>>>>>>>> >>>>>>>>> On Tue, Mar 13, 2018 at 11:21 AM, Wes McKinney <wesmck...@gmail.com> >>>>>>>>> wrote: >>>>>>>>>> I fixed ARROW-2227 in https://github.com/apache/arrow/pull/1740, so >>>>>>>>>> if >>>>>>>>>> someone could review that would be great. >>>>>>>>>> >>>>>>>>>> I am going to make sure the C++/Python HDFS tests run locally, then >>>>>>>>>> after the patch above is merged we should be good to make the RC. >>>>>>>>>> >>>>>>>>>> Note to other onlookers -- please feel free to keep writing new >>>>>>>>>> patches not mentioned here. They may just have to go in to the next >>>>>>>>>> release >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> Wes >>>>>>>>>> >>>>>>>>>> On Tue, Mar 13, 2018 at 12:56 AM, Wes McKinney <wesmck...@gmail.com> >>>>>>>>>> wrote: >>>>>>>>>>> Things are looking pretty good. I'm waiting on a build for >>>>>>>>>>> ARROW-1643 >>>>>>>>>>> to go in, and there's a last blocker bug (ARROW-2227) that we ought >>>>>>>>>>> to >>>>>>>>>>> try to fix tomorrow before cutting the RC. I can start the vote >>>>>>>>>>> after >>>>>>>>>>> that if nothing else comes up >>>>>>>>>>> >>>>>>>>>>> I created ARROW-2300 in the course of trying to test ARROW-1643. I >>>>>>>>>>> have an alternative way to run the HDFS tests (since we aren't >>>>>>>>>>> running >>>>>>>>>>> them in Travis CI) so most likely I will try out the HDFS tests and >>>>>>>>>>> then move ARROW-2300 to the next release milestone. >>>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> Wes >>>>>>>>>>> >>>>>>>>>>> On Mon, Mar 12, 2018 at 12:03 AM, Wes McKinney >>>>>>>>>>> <wesmck...@gmail.com> wrote: >>>>>>>>>>>> I've done a pass over the remaining JIRAs -- I think we're going to >>>>>>>>>>>> need at least another full day to get things buttoned up, so I >>>>>>>>>>>> would >>>>>>>>>>>> say we're looking at an RC on Tuesday. >>>>>>>>>>>> >>>>>>>>>>>> In progress: >>>>>>>>>>>> - ARROW-1425: Patch up, needs some editing, may be moved to 0.10.0 >>>>>>>>>>>> - ARROW-2282: Patch up, needs some test cases >>>>>>>>>>>> - ARROW-1974: Patch in review in apache/parquet-cpp -- may want to >>>>>>>>>>>> move this JIRA to the Parquet project >>>>>>>>>>>> - ARROW-2122: Patch to be reviewed >>>>>>>>>>>> - ARROW-2135: Patch to be merged >>>>>>>>>>>> >>>>>>>>>>>> TODO >>>>>>>>>>>> - ARROW-2082: Parquet segfault <- to be investigated, may be moved >>>>>>>>>>>> to 0.10.0 >>>>>>>>>>>> - ARROW-2118: Fix rough edge with reading length-0 files >>>>>>>>>>>> - ARROW-2227: Bug with creating chunked arrays in Table.from_pandas >>>>>>>>>>>> - ARROW-2292: Deprecation / renaming a Python method >>>>>>>>>>>> >>>>>>>>>>>> Out of the two items in TODO, ARROW-2118 and ARROW-2292 are small >>>>>>>>>>>> matters, so I will take care of them. ARROW-2227 may not be fixable >>>>>>>>>>>> within ~1 day but it would be useful to have a diagnosis in case >>>>>>>>>>>> the >>>>>>>>>>>> fix is easy -- multiple users hit this bug. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Wes >>>>>>>>>>>> >>>>>>>>>>>> On Thu, Mar 8, 2018 at 8:47 PM, Kouhei Sutou <k...@clear-code.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> Thanks! >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> kou >>>>>>>>>>>>> >>>>>>>>>>>>> In >>>>>>>>>>>>> <cajpuwmbsxxw9-8vrj3xwud_p3y7sosdmc6xc53pkrjusewm...@mail.gmail.com> >>>>>>>>>>>>> "Re: Working towards getting 0.9.0 release candidate up next >>>>>>>>>>>>> week" on Thu, 8 Mar 2018 20:44:14 -0500, >>>>>>>>>>>>> Wes McKinney <wesmck...@gmail.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> hi Kou -- yes, I think this is a good idea. It will require a >>>>>>>>>>>>>> little >>>>>>>>>>>>>> bit of work to be able to produce a viable standalone source >>>>>>>>>>>>>> tarball. >>>>>>>>>>>>>> Between Uwe, Phillip, Antoine, and I, we should be able to come >>>>>>>>>>>>>> up >>>>>>>>>>>>>> with a plan to do this >>>>>>>>>>>>>> >>>>>>>>>>>>>> - Wes >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Thu, Mar 8, 2018 at 8:33 PM, Kouhei Sutou >>>>>>>>>>>>>> <k...@clear-code.com> wrote: >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> - Updating pip packages for C++ and Python >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Can we try adding PyArrow source package to PyPI at the >>>>>>>>>>>>>>> 0.9.0? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I want to install PyArrow with Arrow C++ installed by .deb >>>>>>>>>>>>>>> or .rpm. I want to use both Red Arrow (Ruby bindings) and >>>>>>>>>>>>>>> PyArrow in the same process via PyCall (Ruby library >>>>>>>>>>>>>>> to integrate with Python). In the case, I need to use the >>>>>>>>>>>>>>> same Arrow C++ in both Red Arrow and PyArrow. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Now, there are only binary packages for PyArrow at >>>>>>>>>>>>>>> https://pypi.python.org/pypi/pyarrow . If there is a source >>>>>>>>>>>>>>> package for PyArrow at PyPI, I can install PyArrow with >>>>>>>>>>>>>>> Arrow C++ installed by .deb or .rpm by "pip --no-binary >>>>>>>>>>>>>>> pyarrow". >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Red Arrow can also use Arrow C++ installed by .deb or .rpm. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> kou >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> In >>>>>>>>>>>>>>> <cakhw1aqgv7sn70jewe5qmw00yevx_xg_zjinywgbo00+c4k...@mail.gmail.com> >>>>>>>>>>>>>>> "Re: Working towards getting 0.9.0 release candidate up next >>>>>>>>>>>>>>> week" on Thu, 8 Mar 2018 11:25:32 -0800, >>>>>>>>>>>>>>> Siddharth Teotia <siddha...@dremio.com> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> All, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I plan to get RC out over the weekend or early Monday. Is that >>>>>>>>>>>>>>>> fine with >>>>>>>>>>>>>>>> everybody? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> We have 6 items in progress -- >>>>>>>>>>>>>>>> https://issues.apache.org/jira/projects/ARROW/versions/12341707#release-report-tab-body. >>>>>>>>>>>>>>>> How do people feel about completing these JIRAs by tomorrow? I >>>>>>>>>>>>>>>> am >>>>>>>>>>>>>>>> completely fine with deferring the RC to early next week >>>>>>>>>>>>>>>> (Mon/Tue/Wed) if >>>>>>>>>>>>>>>> necessary. Just looking for consensus. Also, I suggest that we >>>>>>>>>>>>>>>> defer the >>>>>>>>>>>>>>>> ones with TODO status. I will do it later today unless I hear >>>>>>>>>>>>>>>> otherwise. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I was wondering if anyone else is interested in collaborating >>>>>>>>>>>>>>>> for the >>>>>>>>>>>>>>>> post-release tasks. As per >>>>>>>>>>>>>>>> https://github.com/apache/arrow/blob/master/dev/release/RELEASE_MANAGEMENT.md, >>>>>>>>>>>>>>>> following are the high level post-release tasks. Please let me >>>>>>>>>>>>>>>> know if you >>>>>>>>>>>>>>>> would like to take up something. I have written my name >>>>>>>>>>>>>>>> against some of >>>>>>>>>>>>>>>> them. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> - Updating the Arrow Website (Sidd) >>>>>>>>>>>>>>>> - Uploading release artifacts to SVN -- looks like PMC karma >>>>>>>>>>>>>>>> is needed >>>>>>>>>>>>>>>> to do this >>>>>>>>>>>>>>>> - Announcing release (Sidd) >>>>>>>>>>>>>>>> - Updating website with new API documentation (Sidd) >>>>>>>>>>>>>>>> - Updating pip packages for C++ and Python >>>>>>>>>>>>>>>> - Updating conda packages for C++ and Python (Sidd) >>>>>>>>>>>>>>>> - Updating Java Maven artifacts in Maven central (Sidd) >>>>>>>>>>>>>>>> - Release blog post >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> If anything is missing, please add to the above list. It will >>>>>>>>>>>>>>>> be helpful >>>>>>>>>>>>>>>> for tracking. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Sidd >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Sun, Mar 4, 2018 at 12:34 PM, Wes McKinney >>>>>>>>>>>>>>>> <wesmck...@gmail.com> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> hey Sidd, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> The Python backlog is still in pretty rough shape. I'd like >>>>>>>>>>>>>>>>> to see if >>>>>>>>>>>>>>>>> we can make an RC by Friday but if not we can defer to >>>>>>>>>>>>>>>>> Monday/Tuesday >>>>>>>>>>>>>>>>> the following week (3/12 or 13). I will trim as much as >>>>>>>>>>>>>>>>> possible out >>>>>>>>>>>>>>>>> of the current backlog to get things down to the essential >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> - Wes >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Sun, Feb 25, 2018 at 11:58 AM, Siddharth Teotia >>>>>>>>>>>>>>>>> <siddha...@dremio.com> >>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> Sounds good. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>>>>> Sidd >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Feb 24, 2018 6:24 PM, "Wes McKinney" >>>>>>>>>>>>>>>>>> <wesmck...@gmail.com> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hi Sidd, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I think we have too many bugs to make an RC this coming >>>>>>>>>>>>>>>>>> week. I suggest >>>>>>>>>>>>>>>>> we >>>>>>>>>>>>>>>>>> defer to the following week. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>>>>> Wes >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Feb 24, 2018 7:09 PM, "Siddharth Teotia" >>>>>>>>>>>>>>>>>> <siddha...@dremio.com> >>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> We currently have 10 issues in progress and PRs are >>>>>>>>>>>>>>>>>> available for 8 of >>>>>>>>>>>>>>>>>> them. In interest of getting a release candidate next week, >>>>>>>>>>>>>>>>>> I would >>>>>>>>>>>>>>>>> request >>>>>>>>>>>>>>>>>> people to review PRs as soon as they can to help make >>>>>>>>>>>>>>>>>> progress and close >>>>>>>>>>>>>>>>>> out as many JIRAs as we can. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> There are 32 issues in TODO list and 25 of them are not yet >>>>>>>>>>>>>>>>>> assigned. I >>>>>>>>>>>>>>>>> am >>>>>>>>>>>>>>>>>> planning to defer some of the unassigned ones later today or >>>>>>>>>>>>>>>>>> tomorrow. It >>>>>>>>>>>>>>>>>> would be good to soon grab/assign the issues that people >>>>>>>>>>>>>>>>>> want to be fixed >>>>>>>>>>>>>>>>>> for 0.9.0. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Here is the link to backlog: >>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/projects/ARROW/versions/12341707 >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> Sidd >>>>>>>>>>>>>>>>> >>>