[jira] [Created] (ARROW-7979) [C++] Implement experimental buffer compression in IPC messages

2020-03-01 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-7979: --- Summary: [C++] Implement experimental buffer compression in IPC messages Key: ARROW-7979 URL: https://issues.apache.org/jira/browse/ARROW-7979 Project: Apache Arrow

[jira] [Created] (ARROW-7978) [Developer] GitHub Actions "lint" task is running include-what-you-use and failing

2020-03-01 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-7978: --- Summary: [Developer] GitHub Actions "lint" task is running include-what-you-use and failing Key: ARROW-7978 URL: https://issues.apache.org/jira/browse/ARROW-7978

[jira] [Created] (ARROW-7977) [C++] Rename fs::FileStats to fs::FileStat

2020-03-01 Thread Kouhei Sutou (Jira)
Kouhei Sutou created ARROW-7977: --- Summary: [C++] Rename fs::FileStats to fs::FileStat Key: ARROW-7977 URL: https://issues.apache.org/jira/browse/ARROW-7977 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-7976) [C++] Add field to IpcOptions to include padding in Buffer metadata accounting

2020-03-01 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-7976: --- Summary: [C++] Add field to IpcOptions to include padding in Buffer metadata accounting Key: ARROW-7976 URL: https://issues.apache.org/jira/browse/ARROW-7976 Project:

[jira] [Created] (ARROW-7975) [C++] Do not include padding bytes in "Buffer" IPC metadata accounting

2020-03-01 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-7975: --- Summary: [C++] Do not include padding bytes in "Buffer" IPC metadata accounting Key: ARROW-7975 URL: https://issues.apache.org/jira/browse/ARROW-7975 Project: Apache

[jira] [Created] (ARROW-7974) [Developer][C++] ResourceWarning in "make check-format"

2020-03-01 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-7974: --- Summary: [Developer][C++] ResourceWarning in "make check-format" Key: ARROW-7974 URL: https://issues.apache.org/jira/browse/ARROW-7974 Project: Apache Arrow

[jira] [Created] (ARROW-7973) [Developer][C++] ResourceWarnings in run_cpplint.py

2020-03-01 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-7973: --- Summary: [Developer][C++] ResourceWarnings in run_cpplint.py Key: ARROW-7973 URL: https://issues.apache.org/jira/browse/ARROW-7973 Project: Apache Arrow

Re: [DISCUSS] Adding "trivial" buffer compression option to IPC protocol (ARROW-300)

2020-03-01 Thread Wes McKinney
On Sun, Mar 1, 2020 at 3:14 PM Antoine Pitrou wrote: > > > Le 01/03/2020 à 22:01, Wes McKinney a écrit : > > In the context of a "next version of the Feather format" ARROW-5510 > > (which is consumed only by Python and R at the moment), I have been > > looking at compressing buffers using fast

Re: [DISCUSS] Adding "trivial" buffer compression option to IPC protocol (ARROW-300)

2020-03-01 Thread Neville Dipale
I also support compression at the buffer level, and making it an extra message. Talking about compression and flight, has anyone tested using grpc's compression to compress at the transport level (if that's a correct way to describe it)? I believe only gzip and brotli are currently supported, so

Re: [DISCUSS] Adding "trivial" buffer compression option to IPC protocol (ARROW-300)

2020-03-01 Thread Antoine Pitrou
Le 01/03/2020 à 22:01, Wes McKinney a écrit : > In the context of a "next version of the Feather format" ARROW-5510 > (which is consumed only by Python and R at the moment), I have been > looking at compressing buffers using fast compressors like ZSTD when > writing the RecordBatch bodies. This

Re: [DISCUSS] Adding "trivial" buffer compression option to IPC protocol (ARROW-300)

2020-03-01 Thread Wes McKinney
On Sun, Mar 1, 2020 at 3:01 PM Wes McKinney wrote: > > In the context of a "next version of the Feather format" ARROW-5510 > (which is consumed only by Python and R at the moment), I have been > looking at compressing buffers using fast compressors like ZSTD when > writing the RecordBatch bodies.

[DISCUSS] Adding "trivial" buffer compression option to IPC protocol (ARROW-300)

2020-03-01 Thread Wes McKinney
In the context of a "next version of the Feather format" ARROW-5510 (which is consumed only by Python and R at the moment), I have been looking at compressing buffers using fast compressors like ZSTD when writing the RecordBatch bodies. This could be handled privately as an implementation detail

[jira] [Created] (ARROW-7972) Allow reading CSV in chunks

2020-03-01 Thread Bulat Yaminov (Jira)
Bulat Yaminov created ARROW-7972: Summary: Allow reading CSV in chunks Key: ARROW-7972 URL: https://issues.apache.org/jira/browse/ARROW-7972 Project: Apache Arrow Issue Type: New Feature

Re: [NIGHTLY] Arrow Build Report for Job nightly-2020-02-29-0

2020-03-01 Thread Krisztián Szűcs
On Sat, Feb 29, 2020 at 3:57 PM Neal Richardson wrote: > > I'm looking into the R failures (https://github.com/apache/arrow/pull/6509). > Since all of those docker-compose jobs are failing on Crossbow on Azure, > but the one that we run on push/pull_request on GHA is passing ( >

[jira] [Created] (ARROW-7971) Create rowcount utility in Rust

2020-03-01 Thread Ken Suenobu (Jira)
Ken Suenobu created ARROW-7971: -- Summary: Create rowcount utility in Rust Key: ARROW-7971 URL: https://issues.apache.org/jira/browse/ARROW-7971 Project: Apache Arrow Issue Type: New Feature