Hi Wes, sure. I opened a ticket and will do a pull request.
Cheers,
--
Animesh
On Fri, Nov 30, 2018 at 5:28 PM Wes McKinney wrote:
> hi Animesh -- can you link to JIRA issues about the C++ improvements
> you're describing? Want to make sure this doesn't fall through the
> cracks
>
> Thanks
>
hi Animesh -- can you link to JIRA issues about the C++ improvements
you're describing? Want to make sure this doesn't fall through the
cracks
Thanks
Wes
On Mon, Nov 26, 2018 at 7:54 AM Antoine Pitrou wrote:
>
>
> Hi Animesh,
>
> Le 26/11/2018 à 14:23, Animesh Trivedi a écrit :
> >
> > * C++
Hi Animesh,
Le 26/11/2018 à 14:23, Animesh Trivedi a écrit :
>
> * C++ bitmap code can be optimized further by using the unsigned integers
> than "int64_t" for bitmap checks, and eliminating the kBitmap. See here
> https://godbolt.org/z/deq0_q - compare the size of the assembly code. And
> the
the hardware speed
>> >> reading/writing
>> >> > > > > rates. As
>> >> > > > > > > > Wes pointed out that a C++ library implementation should
>> be
>> >> > > memory-IO
>> >> > > > > >
ype of
>> > > > > > > > > semantic, create a movement technique that depends on
>> that.
>> > > Arrow
>> > > > > > > doesn't
>> > > > > > > > > force a particular API since the data itself is defined
>> by its
>> > >
gt; raw
> > > > > pointers.
> > > > > > > > > >
> > > > > > > > > > I'm looking at your code
> > > > > > > > > >
> > > > > > > > > > private void consumeFloat4(FieldV
checksum+=accessor.get(i);
> > > > > > > > > }
> > > > > > > > > }
> > > > > > > > > }
> > > > > > > > >
> > > > > > > > > You'll want to get a Java-Arrow expert from Dremio to advise
> > you
> > > &
ructures in its runtime rather
> > than
> > > > > > > copying? If not, how are Crail's runtime data structures
> > different?
> > > > > > >
> > > > > > > - Wes
> > > > > > > On Wed, Sep 19, 2018 at
t; > > > > itself). Especially when accessing the data through Arrow's raw data
> > > > > pointers (and using for example std::string_view-like constructs).
> > > > > >
> > > > > > In C/C++ the fast data structures ar
Netty
> > > layer (but please correct me if I'm wrong, I'm not an expert on this).
> > > Those calls are the only reason I can think of that would degrade the
> > > performance a bit compared to a pure JAva case. I don't know if the
> > Unsafe
> > > ca
; > I don't have a similar machine so it's not easy to relate my numbers to
> > yours, but if you can get that data consumed with 100 Gbps in a pure Java
> > case, I don't see any reason (resulting from Arrow format / off-heap
> > storage) why you wouldn't be able to get at least really close. Can you
&g
h your consumption
> functions in the first place?
> >
> >
> > I'm interested to see your progress on this.
> >
> >
> > Kind regards,
> >
> >
> > Johan Peltenburg
> >
> >
> > From: Animesh Tr
ve arrays in Java with your consumption functions
> in the first place?
>
>
> I'm interested to see your progress on this.
>
>
> Kind regards,
>
>
> Johan Peltenburg
>
> ________
> From: Animesh Trivedi
> Sent: Wednesday, September 19,
...@crail.apache.org
Subject: [JAVA] Arrow performance measurement
Hi all,
A week ago, Wes and I had a discussion about the performance of the
Arrow/Java implementation on the Apache Crail (Incubating) mailing list (
http://mail-archives.apache.org/mod_mbox/crail-dev/201809.mbox/browser). In
a nutshell: I am
Hi all,
A week ago, Wes and I had a discussion about the performance of the
Arrow/Java implementation on the Apache Crail (Incubating) mailing list (
http://mail-archives.apache.org/mod_mbox/crail-dev/201809.mbox/browser). In
a nutshell: I am investigating the performance of various file formats
15 matches
Mail list logo
