[
https://issues.apache.org/jira/browse/KAFKA-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016113#comment-14016113
]
Stephan Lachowsky edited comment on KAFKA-1456 at 6/3/14 1:05 AM:
------------------------------------------------------------------
Hello all,
First off, I would really like to see this functionality get added, but I'd
like to make sure the wire protocol is done properly, before it is picked by a
release and there is no going back.
Here are, in my estimation, the issues with the current implementation:
- LZ4 and LZ4HC generate the same output format, so they shouldn't have
different compression codec enums... this is a producer configuration issue
only. The same caveat applies to the compressor "level" parameter, it is
basically a producer CPU/compression tradeoff.
- The LZ4Block format used by net.jpountz.lz4.LZ4BlockInputStream and
net.jpountz.lz4.LZ4BlockOutputStream is a block based format that isn't well
documented outside of the java code. I would recommend that something
documented be used, like the format defined by the LZ4 author:
http://fastcompression.blogspot.com/2013/04/lz4-streaming-format-final.html
was (Author: stephanl):
Hello all,
First off, I would really like to see this functionality get added, but I'd
like to make sure the wire protocol is done properly, before it is picked by a
release and there is no going back.
Here are, in my estimation, the issues with the current implementation:
- LZ4 and LZ4HC generate the same output format, so they shouldn't have
different compression codec enums... this is a producer configuration issue
only. The same caveat applies to the compressor "level" parameter, it is
basically a producer CPU/compression tradeoff.
- The LZ4Block format used by net.jpountz.lz4.LZ4BlockInputStream and
net.jpountz.lz4.LZ4BlockOutputStream is a block based format that isn't well
documented outside of the java code. I would recommend that something
documented by used, like the format defined by the LZ4 author:
http://fastcompression.blogspot.com/2013/04/lz4-streaming-format-final.html
> Add LZ4 and LZ4C as a compression codec
> ---------------------------------------
>
> Key: KAFKA-1456
> URL: https://issues.apache.org/jira/browse/KAFKA-1456
> Project: Kafka
> Issue Type: Improvement
> Reporter: Joe Stein
> Labels: newbie
> Fix For: 0.8.2
>
> Attachments: KAFKA-1456.patch, KAFKA-1456_2014-05-19_15:01:10.patch,
> KAFKA-1456_2014-05-19_16:39:01.patch, KAFKA-1456_2014-05-19_18:19:32.patch,
> KAFKA-1456_2014-05-19_23:24:27.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
