Re: [VOTE][Format] Add in a new interval type can combines Month, Days and Nanoseconds

2021-08-17 Thread Fan Liya
+1 On Wed, Aug 18, 2021 at 8:28 AM Keith Kraus wrote: > +1 (non-binding) > > On Tue, Aug 17, 2021 at 7:34 PM Jorge Cardoso Leitão < > jorgecarlei...@gmail.com> wrote: > > > +1 > > > > On Tue, Aug 17, 2021 at 8:50 PM Micah Kornfield > > wrote: > > > > > Hello, > > > As discussed previously [1],

Re: [VOTE][Format] Add in a new interval type can combines Month, Days and Nanoseconds

2021-08-17 Thread Keith Kraus
+1 (non-binding) On Tue, Aug 17, 2021 at 7:34 PM Jorge Cardoso Leitão < jorgecarlei...@gmail.com> wrote: > +1 > > On Tue, Aug 17, 2021 at 8:50 PM Micah Kornfield > wrote: > > > Hello, > > As discussed previously [1], I'd like to call a vote to add a new > interval > > type which is a triple of

Re: [VOTE][Format] Add in a new interval type can combines Month, Days and Nanoseconds

2021-08-17 Thread Jorge Cardoso Leitão
+1 On Tue, Aug 17, 2021 at 8:50 PM Micah Kornfield wrote: > Hello, > As discussed previously [1], I'd like to call a vote to add a new interval > type which is a triple of Month, Days, and nanoseconds. The formal > definition is defined in a PR [2] along with Java and C++ implementations >

[VOTE][Format] Add in a new interval type can combines Month, Days and Nanoseconds

2021-08-17 Thread Micah Kornfield
Hello, As discussed previously [1], I'd like to call a vote to add a new interval type which is a triple of Month, Days, and nanoseconds. The formal definition is defined in a PR [2] along with Java and C++ implementations that have been verified with integration tests. The PR has gone through

Flight SQL

2021-08-17 Thread Kyle Porter
Hello All, We've been working on adding Flight SQL as a formal part of Arrow Flight and the PR is available for review at https://github.com/apache/arrow/pull/10906. Most of the spec is now in place and, if you are interested, it would be good to have more eyes on the PR. Thanks! *Kyle Porter*

Re: [DISCUSS] Developing an "Arrow Compute IR [Intermediate Representation]" to decouple language front ends from Arrow-native compute engines

2021-08-17 Thread Phillip Cloud
On Tue, Aug 17, 2021 at 10:56 AM Wes McKinney wrote: > Looking at Ben's alternate PR [1], having an IR that leans heavily on > memory references to an out-of-band data sidecar seems like an > approach that would substantially ratchet up the implementation > complexity as producing the IR would

Re: [DISCUSS] Developing an "Arrow Compute IR [Intermediate Representation]" to decouple language front ends from Arrow-native compute engines

2021-08-17 Thread Benjamin Kietzman
WRT out-of-band data: if encapsulation is the priority over reuse of Buffer etc that's straightforward to accommodate by replacement with an alternative to Buffer. I have made that change to my PR in https://github.com/apache/arrow/pull/10934/commits/ebd4fc665579dd6bba29c5c4731c2350ea0fa70a > as

Re: [DISCUSS][Python] Making NumPy optional dependency?

2021-08-17 Thread Joris Van den Bossche
On Tue, 17 Aug 2021 at 16:20, Alessandro Molina wrote: > ... > There are by the way some interesting points, like the fact that the mask > for a pyarrow array can only be a numpy array, how could I create a masked > array without numpy? I guess that accepting arrow arrays as mask is > actually

Re: [DISCUSS] Developing an "Arrow Compute IR [Intermediate Representation]" to decouple language front ends from Arrow-native compute engines

2021-08-17 Thread Wes McKinney
Looking at Ben's alternate PR [1], having an IR that leans heavily on memory references to an out-of-band data sidecar seems like an approach that would substantially ratchet up the implementation complexity as producing the IR would then have the level of complexity of producing the Arrow IPC

Re: [DISCUSS][Python] Making NumPy optional dependency?

2021-08-17 Thread Alessandro Molina
I did a quick and dirty experiment and what I got was a segmentation fault, but I guess that a lot depends on what of the things you are using was inlined at compile time. I could get through the segfault by using PyImport_ImportModule to check if numpy exists and got a working minimal case where

Re: [DISCUSS] Clarifying interpretation of Time32/Time64 past 24 hours [leap seconds]

2021-08-17 Thread Antoine Pitrou
Neither PostgreSQL, SQL Server nor Oracle seem to support leap seconds at all. So it seems perhaps the Arrow format should not support them either. However, at some IO boundaries (such as when converting from CSV or JSON), we may want to "coerce" leap seconds (which probably means turning