m: Radev, Martin
> Sent: Thursday, July 11, 2019 2:08 PM
> To: dev@arrow.apache.org; emkornfi...@gmail.com
> Cc: Raoofy, Amir ; Karlstetter, Roman <
> roman.karlstet...@tum.de>
> Subject: Re: Adding a new encoding for FP data
>
> Hello Micah,
>
>
> the changes will
All,
Can someone please help me to unsubscribe to this group?
Thank you.
-Original Message-
From: Radev, Martin
Sent: Thursday, July 11, 2019 2:08 PM
To: dev@arrow.apache.org; emkornfi...@gmail.com
Cc: Raoofy, Amir ; Karlstetter, Roman
Subject: Re: Adding a new encoding for FP data
hieve a
> > > different compression ratio and compression/decompression speed. It
> turned
> > > out that when the transformation is applied, zstd would attempt to
> compress
> > > fewer frames and copy the other. This could lead to less attempts to
> build
> > > a huffma
would attempt to
> compress
> > > fewer frames and copy the other. This could lead to less attempts to
> build
> > > a huffman tree. It's hard to pin-point the exact reason.
> > >
> > > I did not try other lossless text compressors but I expect
; For code, I can polish my patches, create a Jira task and submit the
> > patches for review.
> >
> >
> > Regards,
> >
> > Martin
> >
> >
> >
> > From: Fan Liya
> > Sent: Thursday, July 11, 2019 11:32:
t;
> Regards,
>
> Martin
>
>
> ________________
> From: Fan Liya
> Sent: Thursday, July 11, 2019 11:32:53 AM
> To: dev@arrow.apache.org
> Cc: Raoofy, Amir; Karlstetter, Roman
> Subject: Re: Adding a new encoding for FP data
>
> Hi Radev,
>
>
olish my patches, create a Jira task and submit the patches
for review.
Regards,
Martin
From: Fan Liya
Sent: Thursday, July 11, 2019 11:32:53 AM
To: dev@arrow.apache.org
Cc: Raoofy, Amir; Karlstetter, Roman
Subject: Re: Adding a new encoding for FP data
Hi
Hi Radev,
Thanks for the information. It seems interesting.
IMO, Arrow has much to do for data compression. However, it seems there are
some differences for memory data compression and external storage data
compression.
Could you please provide some reference for stream splitting?
Best,
Liya Fan
Hello people,
there has been discussion in the Apache Parquet mailing list on adding a new
encoder for FP data.
The reason for this is that the supported compressors by Apache Parquet (zstd,
gzip, etc) do not compress well raw FP data.
In my investigation it turns out that a very simple simpl
