Re: [VOTE] Move Arrow DataFusion Subproject to new Top Level Apache Project

2024-03-05 Thread Parth Chandra
+1 (non-binding) On Sun, Mar 3, 2024 at 8:04 PM Mehmet Ozan Kabak wrote: > +1 (non-binding) > -- > *Mehmet Ozan Kabak* > Co-founder & CEO @ Synnada, Inc. > > > On Sun, Mar 3, 2024 at 7:33 PM Jacob Wujciak-Jens > wrote: > > > +1 (non-binding) > > > > On Mon, Mar 4, 2024 at 3:39 AM Yang Jiang

Re: [VOTE] Accept donation of Comet Spark native engine

2024-01-29 Thread Parth Chandra
+1 (non-binding) On Sat, Jan 27, 2024 at 7:44 AM Andy Grove wrote: > Hello, > > This vote is to determine if the Arrow PMC is in favor of accepting the > donation of Comet (a Spark native engine that is powered by DataFusion and > the Rust implementation of Arrow). > > The donation was

Re: [DISCUSS] Donation of a Spark native engine based on DataFusion & Arrow

2024-01-11 Thread Parth Chandra
Full disclosure: I worked on the original value vector implementation that became Apache arrow and currently work with Chao, et al on the native engine that is being discussed. I believe that integration of DataFusion with Spark will drive both development and user interest in arrow-rs and

Re: [DISCUSS][JAVA]Support Fast/Unsafe Vector APIs for Arrow

2019-04-30 Thread Parth Chandra
FWIW, in Drill's Value Vector code, we found that bounds checking was a major performance bottleneck in operators that wrote to vectors. Scans, as a result, we particularly affected. Another bottleneck was the zeroing of vectors. There were many unnecessary bounds checks. For example in a varchar

Re: Is there plan to support BigEndian Systems like SUN SPARC Hardware ?

2016-08-04 Thread Parth Chandra
Drill's assumption of little endian is in the ValueVector code, and Arrow has inherited the same assertion. ( https://github.com/apache/arrow/blob/master/java/memory/src/main/java/io/netty/buffer/UnsafeDirectLittleEndian.java#L58 ) In the Java implementation, the underlying Netty implementation

Re: Is there plan to support BigEndian Systems like SUN SPARC Hardware ?

2016-08-01 Thread Parth Chandra
Sorry, please disregard my reply. I misunderstood this to be a question on the Drill mailing list. On Mon, Aug 1, 2016 at 9:50 AM, Parth Chandra <par...@apache.org> wrote: > Short answer is no. > Drill's in memory format assumes little endian and it would be very > disruptive to

Re: Is there plan to support BigEndian Systems like SUN SPARC Hardware ?

2016-08-01 Thread Parth Chandra
Short answer is no. Drill's in memory format assumes little endian and it would be very disruptive to have to change that. There is a JIRA to fix the Drill client to allow little endian and we should fix that, but it is currently low in priority. On Mon, Aug 1, 2016 at 9:27 AM, Sanjay Rao