[jira] [Created] (ARROW-2097) [Python] Suppress valgrind stdout/stderr in Travis CI builds when there are no errors

2018-02-05 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-2097: --- Summary: [Python] Suppress valgrind stdout/stderr in Travis CI builds when there are no errors Key: ARROW-2097 URL: https://issues.apache.org/jira/browse/ARROW-2097

[jira] [Created] (ARROW-2096) [C++] Turn off Boost_DEBUG to trim build output

2018-02-05 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-2096: --- Summary: [C++] Turn off Boost_DEBUG to trim build output Key: ARROW-2096 URL: https://issues.apache.org/jira/browse/ARROW-2096 Project: Apache Arrow Issue

[jira] [Created] (ARROW-2095) [C++] Suppress ORC EP build logging by default

2018-02-05 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-2095: --- Summary: [C++] Suppress ORC EP build logging by default Key: ARROW-2095 URL: https://issues.apache.org/jira/browse/ARROW-2095 Project: Apache Arrow Issue

[jira] [Created] (ARROW-2094) [Python] Use toolchain libraries and PROTOBUF_HOME for protocol buffers

2018-02-05 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-2094: --- Summary: [Python] Use toolchain libraries and PROTOBUF_HOME for protocol buffers Key: ARROW-2094 URL: https://issues.apache.org/jira/browse/ARROW-2094 Project: Apache

Re: [Python] Retrieving a RecordBatch from plasma inside a function

2018-02-05 Thread Philipp Moritz
Hey Alberto, Thanks for your message! I'm trying to reproduce it. Can you attach the code you use to write the batch into the store? Also can you say which version of Python and Arrow you are using? On my installation, I get ``` In [*5*]: plasma.ObjectID(bytearray("keynumber1keynumber1",

[jira] [Created] (ARROW-2093) [Python] Possibly do not test pytorch serialization in Travis CI

2018-02-05 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-2093: --- Summary: [Python] Possibly do not test pytorch serialization in Travis CI Key: ARROW-2093 URL: https://issues.apache.org/jira/browse/ARROW-2093 Project: Apache Arrow

Re: Delta dictionaries: implementation

2018-02-05 Thread Wes McKinney
hi Dimitri, No one is working on it yet in C++, nor have we worked on any API design sketches. I think there may be some work in JavaScript. Please feel free to open some JIRAs and propose APIs / behavior or work on an implementation. Thanks, Wes On Mon, Feb 5, 2018 at 11:37 AM, Dimitri Vorona

[jira] [Created] (ARROW-2092) [Python] Enhance benchmark suite

2018-02-05 Thread Antoine Pitrou (JIRA)
Antoine Pitrou created ARROW-2092: - Summary: [Python] Enhance benchmark suite Key: ARROW-2092 URL: https://issues.apache.org/jira/browse/ARROW-2092 Project: Apache Arrow Issue Type:

[Python] Retrieving a RecordBatch from plasma inside a function

2018-02-05 Thread ALBERTO Bocchinfuso
Good morning, I am experiencing problems with the RecordBatches stored in plasma in a particular situation. If I return a RecordBatch as result of a python function, I am able to read just the metadata, while I get an error when reading the columns. For example, the following code def

Delta dictionaries: implementation

2018-02-05 Thread Dimitri Vorona
Hi, ARROW-1727 added format support for delta dictionaries. It makes possible to interleave record batches which contain dictionary encoded field with delta dictionary batches which add new dictionary entries. As far as I can see there is not implementation of this feature in cpp, yet. Is anyone

Spark DataFrame <--> Arrow Roundtrip

2018-02-05 Thread Michael Shtelma
Hi all, I would like to make some changes (updates) to the data stored in Spark data frames, which I get as a result of different queries. Afterwards, I would like to operate with these changed data frames as with normal data frames in Spark, e.g. use them for further transformations. I would

[jira] [Created] (ARROW-2091) Interacting with arrow/pyarrow in C++

2018-02-05 Thread Jun (JIRA)
Jun created ARROW-2091: -- Summary: Interacting with arrow/pyarrow in C++ Key: ARROW-2091 URL: https://issues.apache.org/jira/browse/ARROW-2091 Project: Apache Arrow Issue Type: Improvement