[jira] [Created] (ARROW-6898) [Java] Fix potential memory leak in ArrowWriter and several test classes

2019-10-15 Thread Ji Liu (Jira)
Ji Liu created ARROW-6898: - Summary: [Java] Fix potential memory leak in ArrowWriter and several test classes Key: ARROW-6898 URL: https://issues.apache.org/jira/browse/ARROW-6898 Project: Apache Arrow

[jira] [Created] (ARROW-6897) Fix MakeArrayFromScalar function validity buffer is null bug

2019-10-15 Thread Chendi.Xue (Jira)
Chendi.Xue created ARROW-6897: - Summary: Fix MakeArrayFromScalar function validity buffer is null bug Key: ARROW-6897 URL: https://issues.apache.org/jira/browse/ARROW-6897 Project: Apache Arrow

Re: [C++] The quest for zero-dependency builds

2019-10-15 Thread Micah Kornfield
Hi Wes, I agree on both accounts that it won't be a done in the short term, and it makes sense to tackle in incrementally. Like I said I don't have much bandwidth at the moment but might be able to re-arrange a few things on my plate. I think some people have asked on the mailing list how they mi

[jira] [Created] (ARROW-6896) [Java] Vector schema root should not share vectors

2019-10-15 Thread Liya Fan (Jira)
Liya Fan created ARROW-6896: --- Summary: [Java] Vector schema root should not share vectors Key: ARROW-6896 URL: https://issues.apache.org/jira/browse/ARROW-6896 Project: Apache Arrow Issue Type: Bug

Re: [Discuss][FlightRPC] Extensions to Flight: "DoBidirectional"

2019-10-15 Thread David Li
Hi Jacques, Thanks for the comments. - I do agree DoExchange is a better name! - FlightData already has metadata fields as a result of prior proposals, so I don't think we need a new message to carry that kind of information. - I like the suggestion of an async handler to handle incoming messages

Re: [Discuss][FlightRPC] Extensions to Flight: "DoBidirectional"

2019-10-15 Thread Jacques Nadeau
I like it. Added some comments to the doc. Might worth discussion here depending on your thoughts. On Tue, Oct 15, 2019 at 7:11 AM David Li wrote: > Hey Ryan, > > Thanks for the comments. > > Concrete example: I've edited the doc to provide a Python strawman. > > Sync vs async: while I don't tou

Re: [Discuss] Streaming: Differentiate between length of RecordBatch and utilized portion-- common use-case?

2019-10-15 Thread John Muehlhausen
A proposal with linked PR now exists in ARROW-5916 and Wes commented that we should kick it around some more. The high-level topic is how Apache Arrow intersects with streaming methodologies: If record batches are strictly immutable, a difficult trade-off is created for streaming data collection:

Re: Looking ahead to 1.0

2019-10-15 Thread John Muehlhausen
ARROW-6837 (which, er, includes ARROW-6836) and ARROW-5916 have PRs. Would appreciate some feedback. I will finish the Python part of 6837 when I know I'm on the right track. Thanks, John On Thu, Oct 10, 2019 at 9:54 AM John Muehlhausen wrote: > The format change is ARROW-6836 ... add a custo

[jira] [Created] (ARROW-6895) parquet::arrow::ColumnReader: ByteArrayDictionaryRecordReader repeats returned values when calling `NextBatch()`

2019-10-15 Thread Adam Hooper (Jira)
Adam Hooper created ARROW-6895: -- Summary: parquet::arrow::ColumnReader: ByteArrayDictionaryRecordReader repeats returned values when calling `NextBatch()` Key: ARROW-6895 URL: https://issues.apache.org/jira/browse/A

Re: [C++] The quest for zero-dependency builds

2019-10-15 Thread Wes McKinney
hi Micah, Definitely Bazel is worth exploring, but we must be realistic about the amount of energy (several hundred hours or more) that's been invested in the build system we have now. So a new build system will be a large endeavor, but hopefully can make things simpler. Aside from the requiremen

Re: Problem running the default example source (Thrift LogicalType)

2019-10-15 Thread Wes McKinney
hi, I tried running the example locally $ g++ reader-writer.cc -larrow -lparquet -isystem . -isystem /home/wesm/local/include -L /home/wesm/local/lib $ ./a.out Parquet Writing and Reading Complete Do you have multiple versions of the Arrow or Parquet libraries installed on your system? If you ca

Re: [Discuss][FlightRPC] Extensions to Flight: "DoBidirectional"

2019-10-15 Thread Ryan Murray
Cool, makes a ton of sense now. Thanks! On Tue, Oct 15, 2019 at 3:11 PM David Li wrote: > Hey Ryan, > > Thanks for the comments. > > Concrete example: I've edited the doc to provide a Python strawman. > > Sync vs async: while I don't touch on it, you could interleave uploads > and downloads if y

Problem running the default example source (Thrift LogicalType)

2019-10-15 Thread Mehran Dashti
Hi, I get the following error when running parquet reader-writer.cc example in example folder of source code: terminate called after throwing an instance of 'parquet::ParquetException' what(): Arrow error: IOError: Metadata contains Thrift LogicalType that is not recognized This problem seems t

[jira] [Created] (ARROW-6894) [C++] Add/document configuration to build with WebAssembly

2019-10-15 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-6894: --- Summary: [C++] Add/document configuration to build with WebAssembly Key: ARROW-6894 URL: https://issues.apache.org/jira/browse/ARROW-6894 Project: Apache Arrow

Re: [Discuss][FlightRPC] Extensions to Flight: "DoBidirectional"

2019-10-15 Thread David Li
Hey Ryan, Thanks for the comments. Concrete example: I've edited the doc to provide a Python strawman. Sync vs async: while I don't touch on it, you could interleave uploads and downloads if you were so inclined. Right now, synchronous APIs make this error-prone, e.g. if both client and server w

[jira] [Created] (ARROW-6893) Missing APT package metadata for versions prior to 0.15.0

2019-10-15 Thread Joshua Randall (Jira)
Joshua Randall created ARROW-6893: - Summary: Missing APT package metadata for versions prior to 0.15.0 Key: ARROW-6893 URL: https://issues.apache.org/jira/browse/ARROW-6893 Project: Apache Arrow

[jira] [Created] (ARROW-6892) [Rust] [DataFusion] Implement optimizer rule to remove redundant projections

2019-10-15 Thread Andy Grove (Jira)
Andy Grove created ARROW-6892: - Summary: [Rust] [DataFusion] Implement optimizer rule to remove redundant projections Key: ARROW-6892 URL: https://issues.apache.org/jira/browse/ARROW-6892 Project: Apache

[jira] [Created] (ARROW-6891) [Rust] [Parquet] Add Utf8 support to ArrowReader

2019-10-15 Thread Andy Grove (Jira)
Andy Grove created ARROW-6891: - Summary: [Rust] [Parquet] Add Utf8 support to ArrowReader Key: ARROW-6891 URL: https://issues.apache.org/jira/browse/ARROW-6891 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-6890) [Rust] [Parquet] ArrowReader fails with seg fault

2019-10-15 Thread Andy Grove (Jira)
Andy Grove created ARROW-6890: - Summary: [Rust] [Parquet] ArrowReader fails with seg fault Key: ARROW-6890 URL: https://issues.apache.org/jira/browse/ARROW-6890 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-6889) [Java] ComplexCopier enable FixedSizeList type & fix RangeEualsVisitor StackOverFlow

2019-10-15 Thread Ji Liu (Jira)
Ji Liu created ARROW-6889: - Summary: [Java] ComplexCopier enable FixedSizeList type & fix RangeEualsVisitor StackOverFlow Key: ARROW-6889 URL: https://issues.apache.org/jira/browse/ARROW-6889 Project: Apache

Re: [Discuss][FlightRPC] Extensions to Flight: "DoBidirectional"

2019-10-15 Thread Ryan Murray
Hey David, I think this proposal makes a lot of sense. I like it and the possibility of remote compute via arrow buffers. One thing that would help me would be a concrete example of the API in a real life use case. Also, what would the client experience be in terms of sync vs asyc? Would the clien

[jira] [Created] (ARROW-6888) [Java] Support copy operation for vector value comparators

2019-10-15 Thread Liya Fan (Jira)
Liya Fan created ARROW-6888: --- Summary: [Java] Support copy operation for vector value comparators Key: ARROW-6888 URL: https://issues.apache.org/jira/browse/ARROW-6888 Project: Apache Arrow Issue T