Re: Possible Arrow 0.15.1 release

2019-10-21 Thread Wes McKinney
I'm still trying to get ARROW-6910 merged which I'd like to have in 0.15.1. I need some help -- I won't be able to spend much time on this in the next 24 hours, so if someone could take over https://github.com/apache/arrow/pull/5701 * The main C++ build is timing out (doesn't seem related to my

Re: [Discuss][FlightRPC] Extensions to Flight: "DoBidirectional"

2019-10-21 Thread David Li
The question is whether to repurpose the existing FlightData structure, and allow for the metadata field to be filled in and data fields to be blank (as a control message), or to wrap the FlightData structure in another structure that explicitly distinguishes between control and data messages.

[jira] [Created] (ARROW-6960) Add information about zstd/lz4 codec installation and linkages for R users

2019-10-21 Thread Grant Nguyen (Jira)
Grant Nguyen created ARROW-6960: --- Summary: Add information about zstd/lz4 codec installation and linkages for R users Key: ARROW-6960 URL: https://issues.apache.org/jira/browse/ARROW-6960 Project:

[jira] [Created] (ARROW-6959) [C++] Clarify what signatures are preferred for compute kernels

2019-10-21 Thread Ben Kietzman (Jira)
Ben Kietzman created ARROW-6959: --- Summary: [C++] Clarify what signatures are preferred for compute kernels Key: ARROW-6959 URL: https://issues.apache.org/jira/browse/ARROW-6959 Project: Apache Arrow

[jira] [Created] (ARROW-6958) [python] tutorial script for arrow in spark throws error

2019-10-21 Thread Karl Svensson (Jira)
Karl Svensson created ARROW-6958: Summary: [python] tutorial script for arrow in spark throws error Key: ARROW-6958 URL: https://issues.apache.org/jira/browse/ARROW-6958 Project: Apache Arrow

[jira] [Created] (ARROW-6957) [CI][Crossbow] Nightly R with sanitizers build fails installing dependencies

2019-10-21 Thread Neal Richardson (Jira)
Neal Richardson created ARROW-6957: -- Summary: [CI][Crossbow] Nightly R with sanitizers build fails installing dependencies Key: ARROW-6957 URL: https://issues.apache.org/jira/browse/ARROW-6957

Re: Possible Arrow 0.15.1 release

2019-10-21 Thread Krisztián Szűcs
All the relevant packaging builds are passing for the current release branch: https://github.com/apache/arrow/pull/5708 Although I've created another branch excluding the following patches [1]: - ARROW-6631: [C++] Do not build any compression libraries by default in C++ build - ARROW-6831: [R]

RE: [ANNOUNCE] New Arrow committer: Eric Erhardt

2019-10-21 Thread Eric Erhardt
Thanks, everyone! I'm looking forward to continue working with you all on this great project. Eric -Original Message- From: Krisztián Szűcs Sent: Friday, October 18, 2019 11:55 AM To: dev Subject: Re: [ANNOUNCE] New Arrow committer: Eric Erhardt Congrats! On Fri, Oct 18, 2019 at

Re: Possible Arrow 0.15.1 release

2019-10-21 Thread Wes McKinney
If that patch is not included then I would guess a number of manual changes will be required to fix builds as the result of the Cython linking changes and the libarrow_python_flight shared library split. On Mon, Oct 21, 2019, 9:45 AM Neal Richardson wrote: > I'd like to propose that

[jira] [Created] (ARROW-6956) [C++] Status should use unique_ptr

2019-10-21 Thread Francois Saint-Jacques (Jira)
Francois Saint-Jacques created ARROW-6956: - Summary: [C++] Status should use unique_ptr Key: ARROW-6956 URL: https://issues.apache.org/jira/browse/ARROW-6956 Project: Apache Arrow

[jira] [Created] (ARROW-6955) [Python] [Packaging] Add Python 3.8 to packaging matrix

2019-10-21 Thread Antoine Pitrou (Jira)
Antoine Pitrou created ARROW-6955: - Summary: [Python] [Packaging] Add Python 3.8 to packaging matrix Key: ARROW-6955 URL: https://issues.apache.org/jira/browse/ARROW-6955 Project: Apache Arrow

[jira] [Created] (ARROW-6954) [Python] [CI] Add Python 3.8 to CI matrix

2019-10-21 Thread Antoine Pitrou (Jira)
Antoine Pitrou created ARROW-6954: - Summary: [Python] [CI] Add Python 3.8 to CI matrix Key: ARROW-6954 URL: https://issues.apache.org/jira/browse/ARROW-6954 Project: Apache Arrow Issue Type:

Re: Possible Arrow 0.15.1 release

2019-10-21 Thread Neal Richardson
I'd like to propose that "ARROW-6631: [C++] Do not build any compression libraries by default in C++ build" and all other cmake-related changes (ARROW-6831, ARROW-6610), be excluded from the patch release. They sound like (build) API changes, not bugfixes, and I fear that including them will cause

Re: Possible Arrow 0.15.1 release

2019-10-21 Thread Antoine Pitrou
If that is not much work for you, then it would be better IMHO. Regards Antoine. Le 21/10/2019 à 18:31, Krisztián Szűcs a écrit : > Because of https://github.com/apache/arrow/pull/5627 > I can apply the exports manually if that is desired. > > > > On Mon, Oct 21, 2019 at 6:26 PM Antoine

Re: Possible Arrow 0.15.1 release

2019-10-21 Thread Krisztián Szűcs
Because of https://github.com/apache/arrow/pull/5627 I can apply the exports manually if that is desired. On Mon, Oct 21, 2019 at 6:26 PM Antoine Pitrou wrote: > > How did you do the cherry-picking? I'm very surprised that you needed > to pick up the Flight additions. > > Regards > >

Re: Possible Arrow 0.15.1 release

2019-10-21 Thread Antoine Pitrou
How did you do the cherry-picking? I'm very surprised that you needed to pick up the Flight additions. Regards Antoine. Le 21/10/2019 à 18:18, Krisztián Szűcs a écrit : > On Mon, Oct 21, 2019 at 5:28 PM Wes McKinney wrote: > >> Thanks Krisztian >> >> I'm going to finish ARROW-6910 soon so

Re: Possible Arrow 0.15.1 release

2019-10-21 Thread Krisztián Szűcs
On Mon, Oct 21, 2019 at 5:28 PM Wes McKinney wrote: > Thanks Krisztian > > I'm going to finish ARROW-6910 soon so it can be merged. I don't have > a patch yet for ARROW-6895 but hope to complete it today, but it is > not a blocker. If you are able to prepare the maintenance branch, it > probably

Re: Possible Arrow 0.15.1 release

2019-10-21 Thread Wes McKinney
Thanks Krisztian I'm going to finish ARROW-6910 soon so it can be merged. I don't have a patch yet for ARROW-6895 but hope to complete it today, but it is not a blocker. If you are able to prepare the maintenance branch, it probably makes sense to check that the binary package builds are looking

Re: Possible Arrow 0.15.1 release

2019-10-21 Thread Krisztián Szűcs
Hi, I've created a small snippet to query the tickets under a specific JIRA version and cherry-pick them in the same order as they were applied to the master branch [1]. A couple new files were added between certain commits, so the patches don't merge cleanly. I'm going to try resolving the

[jira] [Created] (ARROW-6953) [C++][Dataset] Implement Gandiva Filter/Projector in Scanner

2019-10-21 Thread Francois Saint-Jacques (Jira)
Francois Saint-Jacques created ARROW-6953: - Summary: [C++][Dataset] Implement Gandiva Filter/Projector in Scanner Key: ARROW-6953 URL: https://issues.apache.org/jira/browse/ARROW-6953

[jira] [Created] (ARROW-6951) [C++][Dataset] Ensure column projection is passed to ParquetDataFragment

2019-10-21 Thread Francois Saint-Jacques (Jira)
Francois Saint-Jacques created ARROW-6951: - Summary: [C++][Dataset] Ensure column projection is passed to ParquetDataFragment Key: ARROW-6951 URL: https://issues.apache.org/jira/browse/ARROW-6951

[jira] [Created] (ARROW-6952) [C++][Dataset] Ensure expression filter is passed ParquetDataFragment

2019-10-21 Thread Francois Saint-Jacques (Jira)
Francois Saint-Jacques created ARROW-6952: - Summary: [C++][Dataset] Ensure expression filter is passed ParquetDataFragment Key: ARROW-6952 URL: https://issues.apache.org/jira/browse/ARROW-6952

[jira] [Created] (ARROW-6950) [C++][Dataset] Add example/benchmark for reading parquet files with dataset

2019-10-21 Thread Francois Saint-Jacques (Jira)
Francois Saint-Jacques created ARROW-6950: - Summary: [C++][Dataset] Add example/benchmark for reading parquet files with dataset Key: ARROW-6950 URL: https://issues.apache.org/jira/browse/ARROW-6950

Re: [Discuss][FlightRPC] Extensions to Flight: "DoBidirectional"

2019-10-21 Thread Antoine Pitrou
Can one of you explain what is being proposed in non-protobuf terms? Knowledge of protobuf shouldn't be required to use Flight. Regards Antoine. Le 21/10/2019 à 15:46, David Li a écrit : > Oneof doesn't actually change the wire encoding; it would just be > application-level logic. (The

Re: [Discuss][FlightRPC] Extensions to Flight: "DoBidirectional"

2019-10-21 Thread David Li
Oneof doesn't actually change the wire encoding; it would just be application-level logic. (The official guide doesn't even mention it in the encoding docs; I found https://stackoverflow.com/questions/52226409/how-protobuf-encodes-oneof-message-construct as well.) If I follow you, Jacques, then

[jira] [Created] (ARROW-6949) [Java] Fix promotable write to handle nullvectors

2019-10-21 Thread Prudhvi Porandla (Jira)
Prudhvi Porandla created ARROW-6949: --- Summary: [Java] Fix promotable write to handle nullvectors Key: ARROW-6949 URL: https://issues.apache.org/jira/browse/ARROW-6949 Project: Apache Arrow