Re: StreamDecoder zero-copy (?) for pre-framed contiguous Messages

2022-07-01 Thread John Muehlhausen
I found this in reader.h ... can't get to it through the docs unless I missed something? https://arrow.apache.org/docs/search.html?q=ReadRecordBatch ARROW_EXPORT Result> ReadRecordBatch( const std::shared_ptr& schema, const DictionaryMemo* dictionary_memo, const IpcReadOptions& options, io

Re: [VOTE] Accept donation of Flight SQL JDBC driver

2022-07-01 Thread Andrew Lamb
+1 (binding) Thank you for driving this work On Thu, Jun 30, 2022 at 9:28 PM Yibo Cai wrote: > +1 > > > On 7/1/22 04:20, David Li wrote: > > Hello, > > > > This vote is to determine if the Arrow PMC is in favor of accepting the > donation of the Flight SQL JDBC driver. > > > > This process was

StreamDecoder zero-copy (?) for pre-framed contiguous Messages

2022-07-01 Thread John Muehlhausen
If I call `Consume(std::shared_ptr buffer)` and it is already pre-framed to contain (e.g.) an entire RecordBatch Message and nothing else, will it use this Buffer in zero-copy mode when calling my Listener::OnRecordBatchDecoded() implementation? I.e. will data in that RecordBatch refer directly to

Re: [FlightSQL] Structured/Serialized representation of query (like JSON) rather than SQL string possible?

2022-07-01 Thread David Li
I quickly drafted these out (sans implementation so far): https://github.com/apache/arrow/pull/13492 On Thu, Jun 30, 2022, at 21:20, David Li wrote: > Ah - somehow I didn't think of that. Yes, we should just implement it > in the same way prepared statements are already implemented. > > On Thu,

Re: accessing Substrait protobuf Python classes from PyArrow

2022-07-01 Thread Weston Pace
Given that Acero does not do any planner / optimizer type tasks I'm not sure you will find anything like this in arrow-cpp or pyarrow. What you are describing I sometimes refer to as "plan slicing and dicing". I have wondered if we will someday need this in Acero but I fear it is a slippery slope