> > If better compression speed in Java Arrow is desired, then opening a > JIRA + a Github PR would be most welcome.
https://issues.apache.org/jira/browse/ARROW-11901 is the one covering it. On Sat, Jan 22, 2022 at 5:40 AM Antoine Pitrou <[email protected]> wrote: > On Thu, 20 Jan 2022 14:42:25 -0700 > Chris Nuernberger <[email protected]> wrote: > > OK, well, for the record tech.ml.dataset supports three major features > the > > official SDK does not - mmap, JDK-17 and just now compression as a > > user-accessible option during write > > - > file:///home/chrisn/dev/tech.all/tech.ml.dataset/docs/tech.v3.libs.arrow.html. > > > > Other lz4 compressors are faster than apache but regardless zstd gets the > > best compression ratio for the very simple test files I tested. > > If better compression speed in Java Arrow is desired, then opening a > JIRA + a Github PR would be most welcome. > > Regards > > Antoine. > > >
