[jira] [Created] (ARROW-7214) [Python] unpickling a pyarrow table with dictionary fields crashes

2019-11-19 Thread Yevgeni Litvin (Jira)
Yevgeni Litvin created ARROW-7214: - Summary: [Python] unpickling a pyarrow table with dictionary fields crashes Key: ARROW-7214 URL: https://issues.apache.org/jira/browse/ARROW-7214 Project: Apache

Re: MIME type

2019-11-19 Thread Micah Kornfield
I would propose: application/apache-arrow-stream application/apache-arrow-file I'm not attached to those names but I think there should be two different mime-types, since the formats are not interchangeable. On Tue, Nov 19, 2019 at 10:31 PM Sutou Kouhei wrote: > Hi, > > What MIME type should

MIME type

2019-11-19 Thread Sutou Kouhei
Hi, What MIME type should be used for Apache Arrow data? application/arrow? Should we use the same MIME type for IPC Streaming Format[1] and IPC File Format[2]? Or should we use different MIME types for them? [1] https://arrow.apache.org/docs/format/Columnar.html#ipc-streaming-format [2]

[jira] [Created] (ARROW-7213) [Java] Represent a data element of a vector as a tree of ArrowBufPointer

2019-11-19 Thread Liya Fan (Jira)
Liya Fan created ARROW-7213: --- Summary: [Java] Represent a data element of a vector as a tree of ArrowBufPointer Key: ARROW-7213 URL: https://issues.apache.org/jira/browse/ARROW-7213 Project: Apache Arrow

[jira] [Created] (ARROW-7212) "go test -bench=8192 -run=. ./math" fails

2019-11-19 Thread Michael Poole (Jira)
Michael Poole created ARROW-7212: Summary: "go test -bench=8192 -run=. ./math" fails Key: ARROW-7212 URL: https://issues.apache.org/jira/browse/ARROW-7212 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-7211) [Rust] [Parquet] Support writing to byte buffers

2019-11-19 Thread Onur Satici (Jira)
Onur Satici created ARROW-7211: -- Summary: [Rust] [Parquet] Support writing to byte buffers Key: ARROW-7211 URL: https://issues.apache.org/jira/browse/ARROW-7211 Project: Apache Arrow Issue

Re: pyarrow read_csv with different amount of columns per row

2019-11-19 Thread Maarten Ballintijn
Hi Elisa, One option is to preprocess the file and add the missing columns. You can do this using two passes (reading once to determine the number of columns and once writing out the lines filled out to the right number of columns) This does not need to take a lot of memory as you can read line

[jira] [Created] (ARROW-7210) [C++] Scalar cast should support time-based types

2019-11-19 Thread Francois Saint-Jacques (Jira)
Francois Saint-Jacques created ARROW-7210: - Summary: [C++] Scalar cast should support time-based types Key: ARROW-7210 URL: https://issues.apache.org/jira/browse/ARROW-7210 Project: Apache

[jira] [Created] (ARROW-7209) [Python] tests with pandas master are failing now __from_arrow__ support landed in pandas

2019-11-19 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-7209: Summary: [Python] tests with pandas master are failing now __from_arrow__ support landed in pandas Key: ARROW-7209 URL:

[jira] [Created] (ARROW-7208) Arrow using ParquetFile class

2019-11-19 Thread Roelant Stegmann (Jira)
Roelant Stegmann created ARROW-7208: --- Summary: Arrow using ParquetFile class Key: ARROW-7208 URL: https://issues.apache.org/jira/browse/ARROW-7208 Project: Apache Arrow Issue Type: Bug

Re: Disabling Gandiva, Plasma, or other components

2019-11-19 Thread Wes McKinney
The relevant JIRA is https://issues.apache.org/jira/browse/ARROW-6776 This is not a very complex project (changing flags and refactoring for code reuse between the "slim" and "comprehensive" build). If there were interested maintainers, we could even have a "pyarrow-slim" on PyPI. But I cannot

Re: Apache Arrow build with needed dependencies only

2019-11-19 Thread Richard Bachmann
Hello Wes and Sebastien, First off a correction from earlier: It appears I misinterpreted the documentation and thought that 'thirdparty/download_dependencies.sh' would download all dependencies no matter what, which isn't the case. Apologies. We were _originally_ building Arrow with the

[jira] [Created] (ARROW-7207) [Rust] Update Generated Flatbuffer Files

2019-11-19 Thread Neville Dipale (Jira)
Neville Dipale created ARROW-7207: - Summary: [Rust] Update Generated Flatbuffer Files Key: ARROW-7207 URL: https://issues.apache.org/jira/browse/ARROW-7207 Project: Apache Arrow Issue Type:

[NIGHTLY] Arrow Build Report for Job nightly-2019-11-19-0

2019-11-19 Thread Crossbow
Arrow Build Report for Job nightly-2019-11-19-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-11-19-0 Failed Tasks: - conda-osx-clang-py27: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-11-19-0-azure-conda-osx-clang-py27 -

[jira] [Created] (ARROW-7206) avoid string concatenation when calling Preconditions#checkArgument

2019-11-19 Thread stephane campinas (Jira)
stephane campinas created ARROW-7206: Summary: avoid string concatenation when calling Preconditions#checkArgument Key: ARROW-7206 URL: https://issues.apache.org/jira/browse/ARROW-7206 Project:

Re: pyarrow read_csv with different amount of columns per row

2019-11-19 Thread Antoine Pitrou
No, there is no way to load CSV files with irregular dimensions, and we don't have any plans currently to support them. Sorry :-( Regards Antoine. Le 19/11/2019 à 05:54, Micah Kornfield a écrit : > +dev@arrow to see if there is a more definitive answer, but I don't believe > this type of

[jira] [Created] (ARROW-7205) [C++][Gandiva] Implement regexp_matches, regexp_like functions in ganidva

2019-11-19 Thread Projjal Chanda (Jira)
Projjal Chanda created ARROW-7205: - Summary: [C++][Gandiva] Implement regexp_matches, regexp_like functions in ganidva Key: ARROW-7205 URL: https://issues.apache.org/jira/browse/ARROW-7205 Project: