Re: Human-readable version of Arrow Schema?

2019-12-23 Thread Micah Kornfield
> > If we were to make the same kinds of forward/backward compatibility > guarantees as with Flatbuffers it could create a lot of work for > maintainers. Does it pay to follow-up with the flatbuffer project to understand if the forward/backward compatibility guarantees the flatbuffers provide exte

Re: [DISCUSS][Java] Enhance code style checking for Java code

2019-12-23 Thread Micah Kornfield
Hi Liya Fan, Thank you for the PR and starting the discussion. Sorry if I missed it but is there something the PR to prevent these problems from reappearing? My personal preference would be to find a mechanism to enforce code style. Run it once (and accept it might be a large change) and from that

Re: [C++][Compute] RFC: add SIMD support to C++ kernel

2019-12-23 Thread Micah Kornfield
I would lean against adding another library dependency. My main concerns with adding another library dependency are: 1. Supporting it across all of the build tool-chains (using a GCC specific option would be my least favorite approach). 2. Distributed binary size (for wheels at least people seem

[jira] [Created] (ARROW-7469) [C++] Improve division related bit operations

2019-12-23 Thread Liya Fan (Jira)
Liya Fan created ARROW-7469: --- Summary: [C++] Improve division related bit operations Key: ARROW-7469 URL: https://issues.apache.org/jira/browse/ARROW-7469 Project: Apache Arrow Issue Type: Improvem

[jira] [Created] (ARROW-7468) [Python] Fix typos

2019-12-23 Thread Kazuaki Ishizaki (Jira)
Kazuaki Ishizaki created ARROW-7468: --- Summary: [Python] Fix typos Key: ARROW-7468 URL: https://issues.apache.org/jira/browse/ARROW-7468 Project: Apache Arrow Issue Type: Improvement

Re: [FlightRPC] Flight fallback mechanism.

2019-12-23 Thread Wes McKinney
hi Vinay, I think these are application-level concerns -- Flight provides an efficient means to transport datasets on a network. The result of a DoGet or DoPut provides an iterator of record batches which generally may not be that big. How you handle the chunks of data in memory (or put them on di

Re: [DISCUSS][C++] Pointer name aliasing

2019-12-23 Thread Wes McKinney
Hi Liya, In general I think when a development guideline promotes readability it is a good thing. Issues like these may be addressed on a case by case basis, though. best Wes On Sun, Dec 22, 2019 at 8:54 PM Fan Liya wrote: > > IMO, this question relates to something general and fundamental. > >

[jira] [Created] (ARROW-7467) [Java] ComplexCopier does incorrect copy for Map nullable info

2019-12-23 Thread Ji Liu (Jira)
Ji Liu created ARROW-7467: - Summary: [Java] ComplexCopier does incorrect copy for Map nullable info Key: ARROW-7467 URL: https://issues.apache.org/jira/browse/ARROW-7467 Project: Apache Arrow Issue

[FlightRPC] Flight fallback mechanism.

2019-12-23 Thread Vinay Kesarwani
Flight is great way to share data among processes, and holding shared data in-memory. I couldn't find any api or design to handle fall back mechanism in case data is not fitting in memory. Cases: 1- Once memory buffer is nearing full, data should spill over to disk. 2- Spilling over disk or memor