Re: [DISCUSS][Format] Time Interval Changes

2019-03-22 Thread Micah Kornfield
Hi arrow-dev, I just wanted to bump this thread to see if anyone wanted to comment or discuss a path forward. If no one chimes in by Monday evening, could I ask a PMC member to start a vote on Tuesday (I believe a member of the PMC needs to initiate a vote?) I will implement the C++ side once

Re: Arrow Flight protocol/API questions

2019-03-22 Thread David Li
Sorry about that! It should be enabled now, let me know if it doesn't work. Best, David On 3/22/19, Antoine Pitrou wrote: > > I second this request. > > Regards > > Antoine. > > > On Fri, 22 Mar 2019 15:26:26 -0700 > Jacques Nadeau wrote: >> Hey David, thanks for sharing this. Can you add

Re: Arrow Flight protocol/API questions

2019-03-22 Thread Antoine Pitrou
I second this request. Regards Antoine. On Fri, 22 Mar 2019 15:26:26 -0700 Jacques Nadeau wrote: > Hey David, thanks for sharing this. Can you add comment capability to the > doc for reviewers? > > thanks, > Jacques > > > On Fri, Mar 22, 2019 at 1:29 PM David Li wrote: > > > Hi all, >

Re: Arrow Flight protocol/API questions

2019-03-22 Thread Jacques Nadeau
Hey David, thanks for sharing this. Can you add comment capability to the doc for reviewers? thanks, Jacques On Fri, Mar 22, 2019 at 1:29 PM David Li wrote: > Hi all, > > To bring this back up again, we've started experimenting with Flight > for real now, and have some proposals. Including

[Python] The next manylinux specification

2019-03-22 Thread Antoine Pitrou
For those who are interested in discussing it: https://discuss.python.org/t/the-next-manylinux-specification/1043 Regards Antoine.

Re: Arrow Flight protocol/API questions

2019-03-22 Thread David Li
Hi all, To bring this back up again, we've started experimenting with Flight for real now, and have some proposals. Including the justifications, they're a little long, so I've put them on a linked Google doc:

tensorflow-io Arrow Datasets and thoughts on support for tensor columns

2019-03-22 Thread Bryan Cutler
Hi All, Recently I have been working with the TensorFlow SIG-IO community to introduce Apache Arrow based Datasets for bringing Arrow data into TensorFlow. SIG-IO is a community maintained repository focused on input/output support for TF, see https://github.com/tensorflow/io (a lot of formats

Re: Creating Arrays from builders using bitmasks

2019-03-22 Thread Felipe Aramburu
Got it. I think I am just going to use arraydata::make for this for now. Thanks a bundle! Felipe On Fri, Mar 22, 2019 at 8:43 AM Francois Saint-Jacques < fsaintjacq...@gmail.com> wrote: > It is not frowned upon to use the ArrayData::Make classes, you just have to > ensure the order of buffers

[jira] [Created] (ARROW-5000) [Python] Fix deprecation warning from setup.py

2019-03-22 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-5000: --- Summary: [Python] Fix deprecation warning from setup.py Key: ARROW-5000 URL: https://issues.apache.org/jira/browse/ARROW-5000 Project: Apache Arrow Issue

Re: Creating Arrays from builders using bitmasks

2019-03-22 Thread Francois Saint-Jacques
It is not frowned upon to use the ArrayData::Make classes, you just have to ensure the order of buffers matches what the specialized Array class expects (also matching the type you're passing to ArrayData). I'd say it is the "preferred" way if your data is already in the required layout, and the

[jira] [Created] (ARROW-4999) [Doc] Add examples on how to construct with ArrayData::Make instead of builder classes

2019-03-22 Thread Francois Saint-Jacques (JIRA)
Francois Saint-Jacques created ARROW-4999: - Summary: [Doc] Add examples on how to construct with ArrayData::Make instead of builder classes Key: ARROW-4999 URL:

Re: Creating Arrays from builders using bitmasks

2019-03-22 Thread Wes McKinney
hi Felipe -- note you don't have to use the builder classes when you have the exact memory layout already, you can wrap you rmemory in arrow::Buffer and construct the arrays directly. I think it would be useful to add APIs for appending to builders with a bitmap. We don't have them now, though.

Re: Creating Arrays from builders using bitmasks

2019-03-22 Thread Felipe Aramburu
Is there a way to use a builder to be able to provide a bit per value LSB as is described in the documentation? I have this already and it seems silly to convert it to something else so that arrow can then make it the same format as what I had to begin with. I know there is the ArrayData class

Re: Creating Arrays from builders using bitmasks

2019-03-22 Thread Francois Saint-Jacques
Actually, this specific method seems to use a byte per value as you questioned. I think it's worth adding documentation and an explicit warning if it confused me. I'll let bkietz chime in to comment on the usage. François On Fri, Mar 22, 2019 at 10:57 AM Francois Saint-Jacques <

Re: Creating Arrays from builders using bitmasks

2019-03-22 Thread Francois Saint-Jacques
Hello Felipe, it's a bit per value as per memory layout documentation. François On Fri, Mar 22, 2019 at 10:48 AM Felipe Aramburu wrote: > In the builder base class I see this api > > >

[jira] [Created] (ARROW-4998) R package fails to install on OSX

2019-03-22 Thread Jordan Ryda (JIRA)
Jordan Ryda created ARROW-4998: -- Summary: R package fails to install on OSX Key: ARROW-4998 URL: https://issues.apache.org/jira/browse/ARROW-4998 Project: Apache Arrow Issue Type: Bug

Creating Arrays from builders using bitmasks

2019-03-22 Thread Felipe Aramburu
In the builder base class I see this api https://github.com/apache/arrow/blob/ad1697e5d25eeaff5630421f55b0120f45cf0ce1/cpp/src/arrow/array/builder_base.h#L149 // Vector append. Treat each zero byte as a nullzero. If valid_bytes is null // assume all of length bits are valid. void

[jira] [Created] (ARROW-4997) [C#] ArrowStreamReader doesn't consume whole stream and doesn't implement sync read

2019-03-22 Thread Eric Erhardt (JIRA)
Eric Erhardt created ARROW-4997: --- Summary: [C#] ArrowStreamReader doesn't consume whole stream and doesn't implement sync read Key: ARROW-4997 URL: https://issues.apache.org/jira/browse/ARROW-4997

[jira] [Created] (ARROW-4996) [Plasma] There are many log files in /tmp

2019-03-22 Thread Yuhong Guo (JIRA)
Yuhong Guo created ARROW-4996: - Summary: [Plasma] There are many log files in /tmp Key: ARROW-4996 URL: https://issues.apache.org/jira/browse/ARROW-4996 Project: Apache Arrow Issue Type: