[
https://issues.apache.org/jira/browse/KAFKA-1253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13935241#comment-13935241
]
Guozhang Wang commented on KAFKA-1253:
--------------------------------------
With the new code, new performance numbers (1m messages, 1K message size,
ack=1):
Two cases: totally random bits in the message (R), all 1's in the message (A).
Note that with the old producer the block size for GZIP is 512 bytes, while for
new producer GZIP is 16KB. According to my test this seems not affect the
compression rate, but may affect the compression performance.
linger-time 500ms:
-----------------------------------
R-NewProducer-NONE: 94215.1875
R-NewProducer-GZIP: 8750.2844
R-NewProducer-SNAPPY: 38133.0079
R-OldProducer-NONE: 64226.0758
R-OldProducer-GZIP: 6073.1204
R-OldProducer-SNAPPY: 28178.5392
A-NewProducer-NONE: 64754.2576
A-NewProducer-GZIP: 34665.6498
A-NewProducer-SNAPPY: 155062.8004
A-OldProducer-NONE: 59948.4443
A-OldProducer-GZIP: 13014.3939
A-OldProducer-SNAPPY: 45945.3251
-----------------------------------
linger-time 0ms (this only affect new producer):
-----------------------------------
R-NewProducer-NONE: 84459.4595
R-NewProducer-GZIP: 8008.2645
R-NewProducer-SNAPPY: 37605.2948
R-OldProducer-NONE: 59084.1950
R-OldProducer-GZIP: 6128.0142
R-OldProducer-SNAPPY: 27643.4001
A-NewProducer-NONE: 68189.5670
A-NewProducer-GZIP: 34461.3688
A-NewProducer-SNAPPY: 116022.7405
A-OldProducer-NONE: 61214.4956
A-OldProducer-GZIP: 13536.7455
A-OldProducer-SNAPPY: 44529.5454
> Implement compression in new producer
> -------------------------------------
>
> Key: KAFKA-1253
> URL: https://issues.apache.org/jira/browse/KAFKA-1253
> Project: Kafka
> Issue Type: Sub-task
> Components: producer
> Reporter: Jay Kreps
> Assignee: Guozhang Wang
> Attachments: KAFKA-1253.patch, KAFKA-1253_2014-02-21_16:15:21.patch,
> KAFKA-1253_2014-02-21_17:55:52.patch, KAFKA-1253_2014-02-24_13:31:50.patch,
> KAFKA-1253_2014-02-26_17:31:30.patch, KAFKA-1253_2014-03-06_17:48:11.patch,
> KAFKA-1253_2014-03-07_16:34:33.patch, KAFKA-1253_2014-03-10_14:35:56.patch,
> KAFKA-1253_2014-03-10_14:39:58.patch, KAFKA-1253_2014-03-10_15:27:47.patch,
> compression-fix.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
