Re: Gradle hanging on unit test run against ProducerFailureHangingTest-testBrokerFailure

2014-07-25 Thread David Corley
Hey Gouzhang, Yes, I spotted that commit and had updated my working copy to that, but the test is still hanging. If it's any help, the test looks like it's doing _something_ as the CPU usage ramps up significantly and stays there until I kill the process. /Dave On Thu, Jul 24, 2014 at 4:10 PM,

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

2014-07-25 Thread Dmitry Bugaychenko (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074119#comment-14074119 ] Dmitry Bugaychenko commented on KAFKA-1414: --- I'm not a scala expert, but the

[jira] [Commented] (KAFKA-1555) provide strong consistency with reasonable availability

2014-07-25 Thread Joe Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074122#comment-14074122 ] Joe Stein commented on KAFKA-1555: -- The simplification and guarantee provided with ack=-1

[jira] [Comment Edited] (KAFKA-1555) provide strong consistency with reasonable availability

2014-07-25 Thread Joe Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074122#comment-14074122 ] Joe Stein edited comment on KAFKA-1555 at 7/25/14 6:11 AM: --- The

[jira] [Updated] (KAFKA-1542) normal IOException in the new producer is logged as ERROR

2014-07-25 Thread David Corley (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Corley updated KAFKA-1542: Attachment: KAFKA-1542.patch Attaching patch normal IOException in the new producer is logged as

[jira] [Updated] (KAFKA-1542) normal IOException in the new producer is logged as ERROR

2014-07-25 Thread David Corley (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Corley updated KAFKA-1542: Status: Patch Available (was: Open) normal IOException in the new producer is logged as ERROR

[jira] [Commented] (KAFKA-1502) source jar is empty

2014-07-25 Thread David Corley (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074158#comment-14074158 ] David Corley commented on KAFKA-1502: - Can't reproduce this on trunk or on the 0.8.1

[jira] [Commented] (KAFKA-960) Upgrade Metrics to 3.x

2014-07-25 Thread Daniel Compton (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074275#comment-14074275 ] Daniel Compton commented on KAFKA-960: -- What are the plans for Metrics 3.0? I was

[jira] [Commented] (KAFKA-1555) provide strong consistency with reasonable availability

2014-07-25 Thread Jiang Wu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074343#comment-14074343 ] Jiang Wu commented on KAFKA-1555: - Jun, We may need to tolerate more than R-1 failures

[jira] [Commented] (KAFKA-1555) provide strong consistency with reasonable availability

2014-07-25 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074460#comment-14074460 ] Jun Rao commented on KAFKA-1555: Jiang, If you use controlled shutdown, we will make sure

Re: Gradle hanging on unit test run against ProducerFailureHangingTest-testBrokerFailure

2014-07-25 Thread Guozhang Wang
Could you get a thread dump when it hangs? On Thu, Jul 24, 2014 at 11:01 PM, David Corley davidcor...@gmail.com wrote: Hey Gouzhang, Yes, I spotted that commit and had updated my working copy to that, but the test is still hanging. If it's any help, the test looks like it's doing

[jira] [Commented] (KAFKA-960) Upgrade Metrics to 3.x

2014-07-25 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074465#comment-14074465 ] Jun Rao commented on KAFKA-960: --- This is probably going to affect many existing clients. The

[jira] [Commented] (KAFKA-1542) normal IOException in the new producer is logged as ERROR

2014-07-25 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074474#comment-14074474 ] Guozhang Wang commented on KAFKA-1542: -- Thanks for the patch. LGTM. normal

[jira] [Commented] (KAFKA-1477) add authentication layer and initial JKS x509 implementation for brokers, producers and consumer for network communication

2014-07-25 Thread Joe Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074513#comment-14074513 ] Joe Stein commented on KAFKA-1477: -- [~junrao] While I think it make sense to not try to

[jira] [Commented] (KAFKA-1555) provide strong consistency with reasonable availability

2014-07-25 Thread Jiang Wu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074522#comment-14074522 ] Jiang Wu commented on KAFKA-1555: - Jun, In your example, what are tthe replication and

[jira] [Commented] (KAFKA-1555) provide strong consistency with reasonable availability

2014-07-25 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074532#comment-14074532 ] Jun Rao commented on KAFKA-1555: In our current design, a message is considered committed

[jira] [Updated] (KAFKA-1123) Broker IPv6 addresses parsed incorrectly

2014-07-25 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1123: --- Priority: Major (was: Minor) Affects Version/s: (was: 0.8.0) 0.8.2

Re: Review Request 23767: Fix KAFKA-1430

2014-07-25 Thread Guozhang Wang
On July 22, 2014, 5:02 p.m., Jun Rao wrote: core/src/main/scala/kafka/cluster/Partition.scala, lines 120-130 https://reviews.apache.org/r/23767/diff/2/?file=637561#file637561line120 We may not be able to remove the readlock here. The issue is that this method accesses not only

Re: Review Request 23767: Fix KAFKA-1430

2014-07-25 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23767/ --- (Updated July 25, 2014, 4:52 p.m.) Review request for kafka. Bugs:

[jira] [Updated] (KAFKA-1430) Purgatory redesign

2014-07-25 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1430: - Attachment: KAFKA-1430_2014-07-25_09:52:43.patch Purgatory redesign --

[jira] [Commented] (KAFKA-1430) Purgatory redesign

2014-07-25 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074567#comment-14074567 ] Guozhang Wang commented on KAFKA-1430: -- Updated reviewboard

[jira] [Commented] (KAFKA-1542) normal IOException in the new producer is logged as ERROR

2014-07-25 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074579#comment-14074579 ] Jun Rao commented on KAFKA-1542: Thanks for the patch. Perhaps we can just log the

[jira] [Commented] (KAFKA-1542) normal IOException in the new producer is logged as ERROR

2014-07-25 Thread David Corley (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074582#comment-14074582 ] David Corley commented on KAFKA-1542: - Sure. Will revise the patch to get the address

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

2014-07-25 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074625#comment-14074625 ] Jun Rao commented on KAFKA-1414: Thanks for patch v4. Looks good. Some minor comments

Re: Gradle hanging on unit test run against ProducerFailureHangingTest-testBrokerFailure

2014-07-25 Thread David Corley
Sure. Attaching here. On Fri, Jul 25, 2014 at 4:16 PM, Guozhang Wang wangg...@gmail.com wrote: Could you get a thread dump when it hangs? On Thu, Jul 24, 2014 at 11:01 PM, David Corley davidcor...@gmail.com wrote: Hey Gouzhang, Yes, I spotted that commit and had updated my working

Re: Gradle hanging on unit test run against ProducerFailureHangingTest-testBrokerFailure

2014-07-25 Thread Guozhang Wang
Did you miss the attachment? On Fri, Jul 25, 2014 at 10:44 AM, David Corley davidcor...@gmail.com wrote: Sure. Attaching here. On Fri, Jul 25, 2014 at 4:16 PM, Guozhang Wang wangg...@gmail.com wrote: Could you get a thread dump when it hangs? On Thu, Jul 24, 2014 at 11:01 PM, David

[jira] [Updated] (KAFKA-1557) ISR reported by TopicMetadataResponse most of the time doesn't match the Zookeeper information (and the truth)

2014-07-25 Thread Oleg Lvovitch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Lvovitch updated KAFKA-1557: - Attachment: server2.properties server1.properties ISR reported by

[jira] [Created] (KAFKA-1557) ISR reported by TopicMetadataResponse most of the time doesn't match the Zookeeper information (and the truth)

2014-07-25 Thread Oleg Lvovitch (JIRA)
Oleg Lvovitch created KAFKA-1557: Summary: ISR reported by TopicMetadataResponse most of the time doesn't match the Zookeeper information (and the truth) Key: KAFKA-1557 URL:

[jira] [Updated] (KAFKA-1557) ISR reported by TopicMetadataResponse most of the time doesn't match the Zookeeper information (and the truth)

2014-07-25 Thread Oleg Lvovitch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Lvovitch updated KAFKA-1557: - Attachment: BrokenKafkaLink.scala ISR reported by TopicMetadataResponse most of the time

[jira] [Updated] (KAFKA-1557) ISR reported by TopicMetadataResponse most of the time doesn't match the Zookeeper information (and the truth)

2014-07-25 Thread Oleg Lvovitch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Lvovitch updated KAFKA-1557: - Description: TL;DR - after a topic is created, and at least one broker in the ISR is restarted,

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

2014-07-25 Thread Anton Karamanov (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074699#comment-14074699 ] Anton Karamanov commented on KAFKA-1414: 50. in this case `for` used for safe and

[jira] [Comment Edited] (KAFKA-1542) normal IOException in the new producer is logged as ERROR

2014-07-25 Thread David Corley (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074582#comment-14074582 ] David Corley edited comment on KAFKA-1542 at 7/25/14 6:07 PM: --

[jira] [Updated] (KAFKA-1557) ISR reported by TopicMetadataResponse most of the time doesn't match the Zookeeper information (and the truth)

2014-07-25 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1557: -- Component/s: (was: consumer) Labels: newbie++ (was: ) ISR reported by

[jira] [Updated] (KAFKA-1367) Broker topic metadata not kept in sync with ZooKeeper

2014-07-25 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1367: - Labels: newbie++ (was: ) Broker topic metadata not kept in sync with ZooKeeper

[jira] [Commented] (KAFKA-1557) ISR reported by TopicMetadataResponse most of the time doesn't match the Zookeeper information (and the truth)

2014-07-25 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074766#comment-14074766 ] Joel Koshy commented on KAFKA-1557: --- For the first question: I don't think cluster

[jira] [Resolved] (KAFKA-1557) ISR reported by TopicMetadataResponse most of the time doesn't match the Zookeeper information (and the truth)

2014-07-25 Thread Oleg Lvovitch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Lvovitch resolved KAFKA-1557. -- Resolution: Duplicate ISR reported by TopicMetadataResponse most of the time doesn't match

[jira] [Updated] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in all unit tests

2014-07-25 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1420: - Description: This is a follow-up JIRA from KAFKA-1389. There are a bunch of places in the unit

[jira] [Updated] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in all unit tests

2014-07-25 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1420: - Description: This is a follow-up JIRA from KAFKA-1389. There are a bunch of places in the unit

[jira] [Updated] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in unit tests

2014-07-25 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1420: - Summary: Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with

[jira] [Updated] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in unit tests

2014-07-25 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1420: - Description: This is a follow-up JIRA from KAFKA-1389. There are a bunch of places in the unit

[jira] [Commented] (KAFKA-1555) provide strong consistency with reasonable availability

2014-07-25 Thread saurabh agarwal (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075038#comment-14075038 ] saurabh agarwal commented on KAFKA-1555: Jun, I agree that ack=-1 works fine for

[jira] [Commented] (KAFKA-960) Upgrade Metrics to 3.x

2014-07-25 Thread Daniel Compton (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075215#comment-14075215 ] Daniel Compton commented on KAFKA-960: -- The main (only?) benefit for us is fixing the

[jira] [Commented] (KAFKA-960) Upgrade Metrics to 3.x

2014-07-25 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075244#comment-14075244 ] Jun Rao commented on KAFKA-960: --- Yes. Upgrade Metrics to 3.x --

[jira] [Commented] (KAFKA-1555) provide strong consistency with reasonable availability

2014-07-25 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075263#comment-14075263 ] Jun Rao commented on KAFKA-1555: So, you are suggesting to reject a message if current

[jira] [Commented] (KAFKA-960) Upgrade Metrics to 3.x

2014-07-25 Thread Daniel Compton (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075266#comment-14075266 ] Daniel Compton commented on KAFKA-960: -- Probably worth parking for now then. Upgrade