[
https://issues.apache.org/jira/browse/ARROW-11901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17309741#comment-17309741
]
Micah Kornfield commented on ARROW-11901:
-----------------------------------------
As noted in the discussion on ML Airlift seems to only support Raw not framed
(and is slower then JNI).
[[email protected]] for this specific Jira as starting place maybe we
can checkin the benchmarks that you made for LZ4 compression. I think there
a few follow-up issues (not asking any one in particular to work on them):
1. Provide JNI bindings that support framed compression
2. Provide a performant pure java decompression for those that don't want to
use JNI
3. Use the existing LZ4 java bindings for compression.
> [Java] Investigate potential performance improvement of compression codec
> -------------------------------------------------------------------------
>
> Key: ARROW-11901
> URL: https://issues.apache.org/jira/browse/ARROW-11901
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Java
> Reporter: Liya Fan
> Assignee: Benjamin Wilhelm
> Priority: Major
>
> In response to the discussion in
> https://github.com/apache/arrow/pull/8949/files#r588046787
> There are some performance penalties in the implementation of the compression
> codecs (e.g. data copying between heap/off-heap data). We need to revise the
> code to improve the performance.
> We should also provide some benchmarks to validate that the performance
> actually improves.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
