Re: having problem with 0.8 gzip compression

2013-07-09 Thread Jun Rao
Could you run the following command on one of the log files of your topic and attach the output? bin/kafka-run-class.sh kafka.tools.DumpLogSegments --files /tmp/kafka-logs/testtopic-0/.log Thanks, Jun On Tue, Jul 9, 2013 at 3:23 PM, Scott Wang < scott.w...@rumbleentertainme

[jira] [Updated] (KAFKA-967) Use key range in ProducerPerformance

2013-07-09 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-967: Status: Patch Available (was: Open) 1. Add messageKeyRange to ProducerPerfConfig 2. In generatePro

[jira] [Updated] (KAFKA-967) Use key range in ProducerPerformance

2013-07-09 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-967: Attachment: KAFKA-967.v1.patch 1. Add messageKeyRange to ProducerPerfConfig 2. In generateProducerD

[jira] [Issue Comment Deleted] (KAFKA-967) Use key range in ProducerPerformance

2013-07-09 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-967: Comment: was deleted (was: 1. Add messageKeyRange to ProducerPerfConfig 2. In generateProducerData,

[jira] [Created] (KAFKA-967) Use key range in ProducerPerformance

2013-07-09 Thread Guozhang Wang (JIRA)
Guozhang Wang created KAFKA-967: --- Summary: Use key range in ProducerPerformance Key: KAFKA-967 URL: https://issues.apache.org/jira/browse/KAFKA-967 Project: Kafka Issue Type: Improvement

Re: having problem with 0.8 gzip compression

2013-07-09 Thread Scott Wang
Another piece of information, the snappy compression also does not work. Thanks, Scott On Tue, Jul 9, 2013 at 11:07 AM, Scott Wang < scott.w...@rumbleentertainment.com> wrote: > I just try it and it still not showing up, thanks for looking into this. > > Thanks, > Scott > > > On Tue, Jul 9, 201

[jira] [Commented] (KAFKA-915) System Test - Mirror Maker testcase_5001 failed

2013-07-09 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13703827#comment-13703827 ] John Fung commented on KAFKA-915: - Hi Joel, After apply kafka-915-v1.patch (which is to cr

[jira] [Updated] (KAFKA-915) System Test - Mirror Maker testcase_5001 failed

2013-07-09 Thread John Fung (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Fung updated KAFKA-915: Status: Open (was: Patch Available) > System Test - Mirror Maker testcase_5001 failed > ---

[jira] [Commented] (KAFKA-966) Allow high level consumer to 'nak' a message and force Kafka to close the KafkaStream without losing that message

2013-07-09 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13703755#comment-13703755 ] Joel Koshy commented on KAFKA-966: -- One way to accomplish this is to turn off autocommit a

[jira] [Created] (KAFKA-966) Allow high level consumer to 'nak' a message and force Kafka to close the KafkaStream without losing that message

2013-07-09 Thread Chris Curtin (JIRA)
Chris Curtin created KAFKA-966: -- Summary: Allow high level consumer to 'nak' a message and force Kafka to close the KafkaStream without losing that message Key: KAFKA-966 URL: https://issues.apache.org/jira/browse/KA

Re: beta2 => Maven Repository Signed and Staged

2013-07-09 Thread Joe Stein
I agree it is very very low touch since it is only a change when building for publishing and does not affect the jar. I don't have objections and unless anyone else does have objections then I will publish and see maybe we get more errors/issues or 0.8.0-beta1 will be available from maven. I shou

Re: having problem with 0.8 gzip compression

2013-07-09 Thread Scott Wang
I just try it and it still not showing up, thanks for looking into this. Thanks, Scott On Tue, Jul 9, 2013 at 8:06 AM, Jun Rao wrote: > Could you try starting the consumer first (and enable gzip in the > producer)? > > Thanks, > > Jun > > > On Mon, Jul 8, 2013 at 9:37 PM, Scott Wang < > scott.

[jira] [Commented] (KAFKA-559) Garbage collect old consumer metadata entries

2013-07-09 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13703525#comment-13703525 ] Guozhang Wang commented on KAFKA-559: - One minor comment: add one debug() at the entry

[jira] [Work started] (KAFKA-965) merge c39d37e9dd97bf2462ffbd1a96c0b2cb05034bae from 0.8 to trunk

2013-07-09 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-965 started by Jun Rao. > merge c39d37e9dd97bf2462ffbd1a96c0b2cb05034bae from 0.8 to trunk >

[jira] [Updated] (KAFKA-965) merge c39d37e9dd97bf2462ffbd1a96c0b2cb05034bae from 0.8 to trunk

2013-07-09 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-965: -- Status: Patch Available (was: In Progress) > merge c39d37e9dd97bf2462ffbd1a96c0b2cb05034bae from 0.8 to tru

[jira] [Updated] (KAFKA-965) merge c39d37e9dd97bf2462ffbd1a96c0b2cb05034bae from 0.8 to trunk

2013-07-09 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-965: -- Attachment: kafka-965.patch Attach a patch. The following files have to be merged manually. # Unmerged paths: #

[jira] [Created] (KAFKA-965) merge c39d37e9dd97bf2462ffbd1a96c0b2cb05034bae from 0.8 to trunk

2013-07-09 Thread Jun Rao (JIRA)
Jun Rao created KAFKA-965: - Summary: merge c39d37e9dd97bf2462ffbd1a96c0b2cb05034bae from 0.8 to trunk Key: KAFKA-965 URL: https://issues.apache.org/jira/browse/KAFKA-965 Project: Kafka Issue Type: T

Re: having problem with 0.8 gzip compression

2013-07-09 Thread Jun Rao
Could you try starting the consumer first (and enable gzip in the producer)? Thanks, Jun On Mon, Jul 8, 2013 at 9:37 PM, Scott Wang < scott.w...@rumbleentertainment.com> wrote: > No, I did not start the consumer before the producer. I actually started > the producer first and nothing showed u

[jira] [Commented] (KAFKA-559) Garbage collect old consumer metadata entries

2013-07-09 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13703007#comment-13703007 ] Joel Koshy commented on KAFKA-559: -- Thanks for the patch. Overall, looks good. Couple of