[jira] [Created] (ARROW-6474) Provide mechanism for python to write out old format

2019-09-05 Thread Micah Kornfield (Jira)
Micah Kornfield created ARROW-6474: -- Summary: Provide mechanism for python to write out old format Key: ARROW-6474 URL: https://issues.apache.org/jira/browse/ARROW-6474 Project: Apache Arrow

[jira] [Created] (ARROW-6473) [Format] Clarify dictionary encoding edge cases

2019-09-05 Thread Micah Kornfield (Jira)
Micah Kornfield created ARROW-6473: -- Summary: [Format] Clarify dictionary encoding edge cases Key: ARROW-6473 URL: https://issues.apache.org/jira/browse/ARROW-6473 Project: Apache Arrow

Re: [DISCUSS] IPC buffer layout for Null type

2019-09-05 Thread Micah Kornfield
Hi Wes and others, I don't have a sense of where Null arrays get created in the existing code base? Also, do you think it is worth the effort make this backwards compatible. We could in theory tie the buffer count to having the continuation value for alignment. The one area were I'm slightly

Re: Timeline for 0.15.0 release

2019-09-05 Thread Micah Kornfield
Just for reference [1] has a dashboard of the current issues: https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.15.0+Release On Thu, Sep 5, 2019 at 3:43 PM Wes McKinney wrote: > hi all, > > It doesn't seem like we're going to be in a position to release at the > beginning of next week.

Re: [ANNOUNCE] New committers: Ben Kietzman, Kenta Murata, and Neal Richardson

2019-09-05 Thread Micah Kornfield
Congrats everyone. On Thu, Sep 5, 2019 at 7:06 PM Ji Liu wrote: > Congratulations! > > Thanks, > Ji Liu > > > -- > From:Fan Liya > Send Time:2019年9月6日(星期五) 09:28 > To:dev > Subject:Re: [ANNOUNCE] New committers: Ben Kietzman,

Re: New Users on JIRA

2019-09-05 Thread paddy horan
Thanks on both counts Wes! From: Wes McKinney Sent: Thursday, September 5, 2019 10:52 PM To: dev Subject: Re: New Users on JIRA hi Paddy, I keep all the e-mail in Gmail, it's easy to search there. The Pony Mail interface works well too

Re: New Users on JIRA

2019-09-05 Thread Wes McKinney
hi Paddy, I keep all the e-mail in Gmail, it's easy to search there. The Pony Mail interface works well too https://lists.apache.org/list.html?dev@arrow.apache.org To assign issues to new users * Navigate to "JIRA Administration > Projects" in the top right * Click on "Apache Arrow" * Click

New Users on JIRA

2019-09-05 Thread paddy horan
Hi All, I have the same issue again where there is a new user (hengruo) that needs permissions changed so I can assign an issue. I know that this was discussed recently which leads me to another question. How do others find previous conversations in the mailing list archives? I find it

Re: [ANNOUNCE] New committers: Ben Kietzman, Kenta Murata, and Neal Richardson

2019-09-05 Thread Ji Liu
Congratulations! Thanks, Ji Liu -- From:Fan Liya Send Time:2019年9月6日(星期五) 09:28 To:dev Subject:Re: [ANNOUNCE] New committers: Ben Kietzman, Kenta Murata, and Neal Richardson Big congratulations to Ben, Kenta and Neal! Best,

Re: [ANNOUNCE] New committers: Ben Kietzman, Kenta Murata, and Neal Richardson

2019-09-05 Thread Fan Liya
Big congratulations to Ben, Kenta and Neal! Best, Liya Fan On Fri, Sep 6, 2019 at 5:33 AM Wes McKinney wrote: > hi all, > > on behalf of the Arrow PMC, I'm pleased to announce that Ben, Kenta, > and Neal have accepted invitations to become Arrow committers. Welcome > and thank you for all your

[jira] [Created] (ARROW-6472) [Java] ValueVector#accept may has potential cast exception

2019-09-05 Thread Ji Liu (Jira)
Ji Liu created ARROW-6472: - Summary: [Java] ValueVector#accept may has potential cast exception Key: ARROW-6472 URL: https://issues.apache.org/jira/browse/ARROW-6472 Project: Apache Arrow Issue

Re: Timeline for 0.15.0 release

2019-09-05 Thread Wes McKinney
hi all, It doesn't seem like we're going to be in a position to release at the beginning of next week. I hope that one more week of work (or less) will be enough to get us there. Aside from merging the alignment changes, we need to make sure that our packaging jobs required for the release

Re: [PROPOSAL] Consolidate Arrow's CI configuration

2019-09-05 Thread Wes McKinney
hi Krisztian, Anyone who's developing in the project can see that the Buildbot setup is working well (at least for Linux builds) and giving much more timely feedback, which has been very helpful. I'm concerned about the "ursabot" approach for a few reasons: * If we are to centralize our tooling

[jira] [Created] (ARROW-6471) [Python] arrow_to_pandas.cc has separate code paths for populating list values into an object array

2019-09-05 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-6471: --- Summary: [Python] arrow_to_pandas.cc has separate code paths for populating list values into an object array Key: ARROW-6471 URL: https://issues.apache.org/jira/browse/ARROW-6471

[jira] [Created] (ARROW-6470) Segmentation fault when trying to serialzie empty SerializeRecordBatch

2019-09-05 Thread Wamsi Viswanath (Jira)
Wamsi Viswanath created ARROW-6470: -- Summary: Segmentation fault when trying to serialzie empty SerializeRecordBatch Key: ARROW-6470 URL: https://issues.apache.org/jira/browse/ARROW-6470 Project:

[jira] [Created] (ARROW-6469) PyArrow HDFS documentation does not mention HDFS short circuit readings

2019-09-05 Thread Paulo Roberto Cerioni (Jira)
Paulo Roberto Cerioni created ARROW-6469: Summary: PyArrow HDFS documentation does not mention HDFS short circuit readings Key: ARROW-6469 URL: https://issues.apache.org/jira/browse/ARROW-6469

[ANNOUNCE] New committers: Ben Kietzman, Kenta Murata, and Neal Richardson

2019-09-05 Thread Wes McKinney
hi all, on behalf of the Arrow PMC, I'm pleased to announce that Ben, Kenta, and Neal have accepted invitations to become Arrow committers. Welcome and thank you for all your contributions!

[jira] [Created] (ARROW-6468) [C++] Remove unused hashing routines

2019-09-05 Thread Antoine Pitrou (Jira)
Antoine Pitrou created ARROW-6468: - Summary: [C++] Remove unused hashing routines Key: ARROW-6468 URL: https://issues.apache.org/jira/browse/ARROW-6468 Project: Apache Arrow Issue Type: Task

[jira] [Created] (ARROW-6467) [Website] Transition to new .asf.yaml machinery for website publishing

2019-09-05 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-6467: --- Summary: [Website] Transition to new .asf.yaml machinery for website publishing Key: ARROW-6467 URL: https://issues.apache.org/jira/browse/ARROW-6467 Project: Apache

[jira] [Created] (ARROW-6466) [Developer] Refactor integration/integration_test.py into a proper Python package

2019-09-05 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-6466: --- Summary: [Developer] Refactor integration/integration_test.py into a proper Python package Key: ARROW-6466 URL: https://issues.apache.org/jira/browse/ARROW-6466

[DISCUSS] IPC buffer layout for Null type

2019-09-05 Thread Wes McKinney
hi folks, One of the as-yet-untested (in integration tests) parts of the columnar specification is the Null layout. In C++ we additionally implemented this by writing two length-0 "placeholder" buffers in the RecordBatch data header, but since the Null layout has no memory allocated nor any

[jira] [Created] (ARROW-6465) [Python] Improve Windows build instructions

2019-09-05 Thread ARF (Jira)
ARF created ARROW-6465: -- Summary: [Python] Improve Windows build instructions Key: ARROW-6465 URL: https://issues.apache.org/jira/browse/ARROW-6465 Project: Apache Arrow Issue Type: Improvement

Re: [PROPOSAL] Consolidate Arrow's CI configuration

2019-09-05 Thread Antoine Pitrou
Le 05/09/2019 à 15:04, Krisztián Szűcs a écrit : >> >> If going with buildbot, this means that the various build steps need to >> be generic like in Travis-CI (e.g. "install", "setup", "before-test", >> "test", "after-test"...) and their contents expressed outside of the >> buildmaster

[jira] [Created] (ARROW-6464) [Java] Refactor FixedSizeListVector#splitAndTransfer with slice API

2019-09-05 Thread Ji Liu (Jira)
Ji Liu created ARROW-6464: - Summary: [Java] Refactor FixedSizeListVector#splitAndTransfer with slice API Key: ARROW-6464 URL: https://issues.apache.org/jira/browse/ARROW-6464 Project: Apache Arrow

Re: [PROPOSAL] Consolidate Arrow's CI configuration

2019-09-05 Thread Krisztián Szűcs
Hey Antoine, On Thu, Sep 5, 2019 at 2:54 PM Antoine Pitrou wrote: > > Le 05/09/2019 à 14:43, Uwe L. Korn a écrit : > > Hello Krisztián, > > > >> Am 05.09.2019 um 14:22 schrieb Krisztián Szűcs < > szucs.kriszt...@gmail.com>: > >> > >>> * The build configuration is automatically updated on a

Re: [PROPOSAL] Consolidate Arrow's CI configuration

2019-09-05 Thread Antoine Pitrou
Le 05/09/2019 à 14:43, Uwe L. Korn a écrit : > Hello Krisztián, > >> Am 05.09.2019 um 14:22 schrieb Krisztián Szűcs : >> >>> * The build configuration is automatically updated on a merge to master? >>> >> Not yet, but this can be automatized too with buildbot itself. > > This is something I

Re: [PROPOSAL] Consolidate Arrow's CI configuration

2019-09-05 Thread Uwe L. Korn
Hello Krisztián, > Am 05.09.2019 um 14:22 schrieb Krisztián Szűcs : > >> * The build configuration is automatically updated on a merge to master? >> > Not yet, but this can be automatized too with buildbot itself. This is something I would actually like to have before getting rid of the

Re: [PROPOSAL] Consolidate Arrow's CI configuration

2019-09-05 Thread Krisztián Szűcs
Hey Uwe, On Thu, Sep 5, 2019 at 1:49 PM Uwe L. Korn wrote: > Hello Krisztián, > > I like this proposal. CI coverage and response time is a crucial thing for > the health of the project. In general I like the consolidation and local > reproducibility of tge builds. Some questions I wanted to ask

Re: [Discuss][Java] Support conversions between delta vector and partial sum vector

2019-09-05 Thread Fan Liya
Hi Micah, Thanks for your comments. I am aware that you have invested lots of time and effort in reviewing the algorithm related code. We really appreciate it. Thank you so much. I agree with you that the plan document is a good idea. In general, the algorithms are driven by applications, so it

Re: [PROPOSAL] Consolidate Arrow's CI configuration

2019-09-05 Thread Uwe L. Korn
Hello Krisztián, I like this proposal. CI coverage and response time is a crucial thing for the health of the project. In general I like the consolidation and local reproducibility of tge builds. Some questions I wanted to ask to make sure I understand your proposal correctly (hopefully they

[jira] [Created] (ARROW-6463) [C++][Python] Rename arrow::fs::Selector to FileSelector

2019-09-05 Thread Krisztian Szucs (Jira)
Krisztian Szucs created ARROW-6463: -- Summary: [C++][Python] Rename arrow::fs::Selector to FileSelector Key: ARROW-6463 URL: https://issues.apache.org/jira/browse/ARROW-6463 Project: Apache Arrow

[jira] [Created] (ARROW-6462) [C++] Can't build with bundled double-conversion on CentOS 6 x86_64

2019-09-05 Thread Sutou Kouhei (Jira)
Sutou Kouhei created ARROW-6462: --- Summary: [C++] Can't build with bundled double-conversion on CentOS 6 x86_64 Key: ARROW-6462 URL: https://issues.apache.org/jira/browse/ARROW-6462 Project: Apache