[jira] [Commented] (KAFKA-992) Double Check on Broker Registration to Avoid False NodeExist Exception

2013-08-02 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13727438#comment-13727438 ] Swapnil Ghike commented on KAFKA-992: - - I think I am not completely clear why

[jira] [Comment Edited] (KAFKA-992) Double Check on Broker Registration to Avoid False NodeExist Exception

2013-08-02 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13727438#comment-13727438 ] Swapnil Ghike edited comment on KAFKA-992 at 8/2/13 7:11 AM: - -

[jira] [Updated] (KAFKA-943) Move all configuration key string to constants

2013-08-02 Thread Sam Meder (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Meder updated KAFKA-943: Resolution: Won't Fix Status: Resolved (was: Patch Available) Resolving this issue since the patch

[jira] [Created] (KAFKA-997) Provide a strict verification mode when reading configuration properties

2013-08-02 Thread Sam Meder (JIRA)
Sam Meder created KAFKA-997: --- Summary: Provide a strict verification mode when reading configuration properties Key: KAFKA-997 URL: https://issues.apache.org/jira/browse/KAFKA-997 Project: Kafka

[jira] [Updated] (KAFKA-997) Provide a strict verification mode when reading configuration properties

2013-08-02 Thread Sam Meder (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Meder updated KAFKA-997: Status: Patch Available (was: Open) Provide a strict verification mode when reading configuration

[jira] [Updated] (KAFKA-997) Provide a strict verification mode when reading configuration properties

2013-08-02 Thread Sam Meder (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Meder updated KAFKA-997: Attachment: strict-verification.patch Add strict verification property and use it in tests and similar

[jira] [Updated] (KAFKA-992) Double Check on Broker Registration to Avoid False NodeExist Exception

2013-08-02 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-992: Description: The current behavior of zookeeper for ephemeral nodes is that session expiration and

[jira] [Commented] (KAFKA-955) After a leader change, messages sent with ack=0 are lost

2013-08-02 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13727824#comment-13727824 ] Jun Rao commented on KAFKA-955: --- Thanks for patch v2. Some more comments. 20.

[jira] [Comment Edited] (KAFKA-989) Race condition shutting down high-level consumer results in spinning background thread

2013-08-02 Thread Phil Hargett (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13727844#comment-13727844 ] Phil Hargett edited comment on KAFKA-989 at 8/2/13 5:33 PM:

[jira] [Commented] (KAFKA-649) Cleanup log4j logging

2013-08-02 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13727846#comment-13727846 ] Jun Rao commented on KAFKA-649: --- Thanks for patch v5. 50. ConsoleConsumer: There is no need

[jira] [Commented] (KAFKA-955) After a leader change, messages sent with ack=0 are lost

2013-08-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13727910#comment-13727910 ] Guozhang Wang commented on KAFKA-955: - Sorry for the name misleading, I did not shut

Re: Client improvement discussion

2013-08-02 Thread Chris Hogue
These sounds like great steps. A couple of votes and questions: 1. Moving serialization out and basing it all off of byte[] for key and payload makes sense. Echoing a response below, we've ended up doing that in some cases anyway, and the others do a trivial transform to bytes with an Encoder.

[jira] [Updated] (KAFKA-989) Race condition shutting down high-level consumer results in spinning background thread

2013-08-02 Thread Phil Hargett (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Hargett updated KAFKA-989: --- Status: Open (was: Patch Available) Not good enough. Deadlocks because ShutdownableThread.shutdown

Re: Client improvement discussion

2013-08-02 Thread Jay Kreps
Great comments, answers inline! On Fri, Aug 2, 2013 at 12:28 PM, Chris Hogue csho...@gmail.com wrote: These sounds like great steps. A couple of votes and questions: 1. Moving serialization out and basing it all off of byte[] for key and payload makes sense. Echoing a response below, we've

[jira] [Commented] (KAFKA-915) System Test - Mirror Maker testcase_5001 failed

2013-08-02 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13728247#comment-13728247 ] Joel Koshy commented on KAFKA-915: -- +1 on the patch. I actually could not reproduce the

[jira] [Closed] (KAFKA-915) System Test - Mirror Maker testcase_5001 failed

2013-08-02 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy closed KAFKA-915. System Test - Mirror Maker testcase_5001 failed ---

[jira] [Resolved] (KAFKA-915) System Test - Mirror Maker testcase_5001 failed

2013-08-02 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy resolved KAFKA-915. -- Resolution: Fixed System Test - Mirror Maker testcase_5001 failed

[jira] [Updated] (KAFKA-992) Double Check on Broker Registration to Avoid False NodeExist Exception

2013-08-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-992: Attachment: KAFKA-992.v3.patch Thanks for the comments Neha. 1,2,5: Done 3. 3.1 and 3.2/3 are

Re: Client improvement discussion

2013-08-02 Thread Chris Hogue
Thanks for the responses. Additional follow-up inline. On Fri, Aug 2, 2013 at 2:21 PM, Jay Kreps jay.kr...@gmail.com wrote: Great comments, answers inline! On Fri, Aug 2, 2013 at 12:28 PM, Chris Hogue csho...@gmail.com wrote: These sounds like great steps. A couple of votes and questions:

[jira] [Comment Edited] (KAFKA-992) Double Check on Broker Registration to Avoid False NodeExist Exception

2013-08-02 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13728358#comment-13728358 ] Swapnil Ghike edited comment on KAFKA-992 at 8/3/13 1:39 AM: -

Re: Client improvement discussion

2013-08-02 Thread Jay Kreps
Cool. With respect to compression performance, we definitely see the same thing, no debate. Of course if you want to just compress the message payloads you can do that now without needing much help from kafka--just pass in the compressed data. Whether it not it will do much depends on the size of

[jira] [Updated] (KAFKA-527) Compression support does numerous byte copies

2013-08-02 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-527: Description: The data path for compressing or decompressing messages is extremely inefficient. We do

[jira] [Updated] (KAFKA-527) Compression support does numerous byte copies

2013-08-02 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-527: Attachment: java.hprof.no-compression.txt Compression support does numerous byte copies

[jira] [Commented] (KAFKA-527) Compression support does numerous byte copies

2013-08-02 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13728452#comment-13728452 ] Jay Kreps commented on KAFKA-527: - To test performance checkout trunk and do ./sbt package

[jira] [Updated] (KAFKA-615) Avoid fsync on log segment roll

2013-08-02 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-615: Attachment: KAFKA-615-v5.patch Attach updated patch v5. Rebased against trunk and with added support for

compression performance

2013-08-02 Thread Jay Kreps
Chris commented in another thread about the poor compression performance in 0.8, even with snappy. Indeed if I run the linear log write throughput test on my laptop I see 75MB/sec with no compression and 17MB/sec with snappy. This is a little surprising as snappy claims 200MB round-trip