Log.scala

2014-09-26 Thread Manjunath Shivakumar
Hi Guys, Just looking at the following snippet https://github.com/apache/kafka/blob/7847e9c703f3a0b70519666cdb8a6e4c8e37c3a7/core/src/main/scala/kafka/log/Log.scala#L259 Presumably this is when the follower is appending to its own log fetching from the leader. ( appendInfo.firstOffset

[jira] [Created] (KAFKA-1654) Provide a way to override server configuration from command line

2014-09-26 Thread Jarek Jarcec Cecho (JIRA)
Jarek Jarcec Cecho created KAFKA-1654: - Summary: Provide a way to override server configuration from command line Key: KAFKA-1654 URL: https://issues.apache.org/jira/browse/KAFKA-1654 Project:

[jira] [Updated] (KAFKA-1647) Replication offset checkpoints (high water marks) can be lost on hard kills and restarts

2014-09-26 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1647: - Description: We ran into this scenario recently in a production environment. This can happen

[jira] [Updated] (KAFKA-1647) Replication offset checkpoints (high water marks) can be lost on hard kills and restarts

2014-09-26 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1647: - Description: We ran into this scenario recently in a production environment. This can happen

[jira] [Commented] (KAFKA-1577) Exception in ConnectionQuotas while shutting down

2014-09-26 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149780#comment-14149780 ] Sriharsha Chintalapani commented on KAFKA-1577: --- [~nehanarkhede] [~jjkoshy]

[jira] [Commented] (KAFKA-1652) NoSuchElementException on server shutdown

2014-09-26 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149783#comment-14149783 ] Neha Narkhede commented on KAFKA-1652: -- Thanks [~sriharsha]. Appreciate the

[jira] [Commented] (KAFKA-1577) Exception in ConnectionQuotas while shutting down

2014-09-26 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149798#comment-14149798 ] Joel Koshy commented on KAFKA-1577: --- Oh now I remember this - the root cause is as

[jira] [Commented] (KAFKA-1577) Exception in ConnectionQuotas while shutting down

2014-09-26 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149803#comment-14149803 ] Joel Koshy commented on KAFKA-1577: --- I think selectNow is a good option. Exception in

[jira] [Commented] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-09-26 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149875#comment-14149875 ] Jay Kreps commented on KAFKA-1642: -- Yeah a sample program that reproduces the issue would

[jira] [Commented] (KAFKA-1577) Exception in ConnectionQuotas while shutting down

2014-09-26 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149888#comment-14149888 ] Joel Koshy commented on KAFKA-1577: --- Hmm.. that is really weird - since the swallow

[jira] [Commented] (KAFKA-1577) Exception in ConnectionQuotas while shutting down

2014-09-26 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149934#comment-14149934 ] Sriharsha Chintalapani commented on KAFKA-1577: --- [~nehanarkhede] I am not

[jira] [Commented] (KAFKA-1577) Exception in ConnectionQuotas while shutting down

2014-09-26 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149966#comment-14149966 ] Neha Narkhede commented on KAFKA-1577: -- [~jjkoshy], [~sriharsha] I remember I tried

[jira] [Updated] (KAFKA-1654) Provide a way to override server configuration from command line

2014-09-26 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1654: - Reviewer: Neha Narkhede Provide a way to override server configuration from command line

[jira] [Commented] (KAFKA-1654) Provide a way to override server configuration from command line

2014-09-26 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1414#comment-1414 ] Neha Narkhede commented on KAFKA-1654: -- Yes, I think that's useful. Would you like to

[jira] [Assigned] (KAFKA-1305) Controller can hang on controlled shutdown with auto leader balance enabled

2014-09-26 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani reassigned KAFKA-1305: - Assignee: Sriharsha Chintalapani (was: Neha Narkhede) Controller can

[jira] [Resolved] (KAFKA-1586) support sticky partitioning in the new producer

2014-09-26 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede resolved KAFKA-1586. -- Resolution: Won't Fix support sticky partitioning in the new producer

[jira] [Commented] (KAFKA-1647) Replication offset checkpoints (high water marks) can be lost on hard kills and restarts

2014-09-26 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150021#comment-14150021 ] Neha Narkhede commented on KAFKA-1647: -- [~jjkoshy] Makes sense, thanks for the

[jira] [Commented] (KAFKA-1654) Provide a way to override server configuration from command line

2014-09-26 Thread Jarek Jarcec Cecho (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150095#comment-14150095 ] Jarek Jarcec Cecho commented on KAFKA-1654: --- Thank you for the feedback

[jira] [Commented] (KAFKA-1618) Exception thrown when running console producer with no port number for the broker

2014-09-26 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150114#comment-14150114 ] Neha Narkhede commented on KAFKA-1618: -- +1 on your latest patch! Exception thrown

[jira] [Updated] (KAFKA-1618) Exception thrown when running console producer with no port number for the broker

2014-09-26 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-1618: - Resolution: Fixed Reviewer: Neha Narkhede (was: Joe Stein) Status: Resolved (was:

Review Request 26107: Patch for KAFKA-1577

2014-09-26 Thread Sriharsha Chintalapani
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26107/ --- Review request for kafka. Bugs: KAFKA-1577

[jira] [Updated] (KAFKA-1577) Exception in ConnectionQuotas while shutting down

2014-09-26 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani updated KAFKA-1577: -- Attachment: KAFKA-1577.patch Exception in ConnectionQuotas while shutting down

[jira] [Updated] (KAFKA-1577) Exception in ConnectionQuotas while shutting down

2014-09-26 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani updated KAFKA-1577: -- Status: Patch Available (was: Reopened) Exception in ConnectionQuotas while

[jira] [Commented] (KAFKA-1577) Exception in ConnectionQuotas while shutting down

2014-09-26 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150129#comment-14150129 ] Sriharsha Chintalapani commented on KAFKA-1577: --- Created reviewboard

Build failed in Jenkins: Kafka-trunk #279

2014-09-26 Thread Apache Jenkins Server
See https://builds.apache.org/job/Kafka-trunk/279/changes Changes: [neha.narkhede] KAFKA-1618 Exception thrown when running console producer with no port number for the broker; reviewed by Neha Narkhede -- Started by an SCM change Building remotely on

Kafka Trunk Branch build failure

2014-09-26 Thread Bhavesh Mistry
Hi Kafka Dev team, It seems latest git pull seems to break trunk branchbuild with following error. is Grandle lib deleted ? ./gradlew clean Error: Could not find or load main class org.gradle.wrapper.GradleWrapperMain Thanks, Bhavesh

Re: Kafka Trunk Branch build failure

2014-09-26 Thread Joe Stein
Hi, yes it was https://issues.apache.org/jira/browse/KAFKA-1490 https://github.com/apache/kafka/tree/trunk#apache-kafka You need to install gradle http://www.gradle.org/installation Then bootstrap and download the wrapper cd kafka_source_dir gradle Now everything else will work

[jira] [Commented] (KAFKA-1499) Broker-side compression configuration

2014-09-26 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150181#comment-14150181 ] Joel Koshy commented on KAFKA-1499: --- That sounds reasonable. Another option would be to

Re: Kafka Trunk Branch build failure

2014-09-26 Thread Bhavesh Mistry
Hi Joe, Thanks for quick response ! Thanks, Bhavesh On Fri, Sep 26, 2014 at 4:19 PM, Joe Stein joe.st...@stealth.ly wrote: Hi, yes it was https://issues.apache.org/jira/browse/KAFKA-1490 https://github.com/apache/kafka/tree/trunk#apache-kafka You need to install gradle

Re: Review Request 26107: Patch for KAFKA-1577

2014-09-26 Thread Neha Narkhede
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26107/#review54755 --- Ship it! Ship It! - Neha Narkhede On Sept. 26, 2014, 10:41

Re: Kafka Trunk Branch build failure

2014-09-26 Thread Bhavesh Mistry
Hi Joe, I run following command. I am consistently get OO memory for ant forked JVM. I am not able to set JVM argument for org.gradle.api.tasks.scala.ScalaDocOptions jvmArgs. Any Help is greatly appreciated. tasks.withType(ScalaDoc) { task scaladocJar(type:Jar) { classifier = 'scaladoc'

[jira] [Updated] (KAFKA-1013) Modify existing tools as per the changes in KAFKA-1000

2014-09-26 Thread Mayuresh Gharat (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayuresh Gharat updated KAFKA-1013: --- Attachment: KAFKA-1013_2014-09-26_18:52:09.patch Modify existing tools as per the changes in

Re: Review Request 26107: Patch for KAFKA-1577

2014-09-26 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26107/#review54760 --- Looks good - we can now actually get rid of the swallowError when

Re: Kafka Trunk Branch build failure

2014-09-26 Thread Bhavesh Mistry
Hi Joe, I have good build now after resolving above issues and other. I had to add *org.gradle.jvmargs=-XX:MaxPermSize=512m *in gradle.properties for OOM and also remove all scala version* 2.9.X* build tasks due to following compilation issues (API not found in 2.9.x version). Please remove the

Re: Review Request 26107: Patch for KAFKA-1577

2014-09-26 Thread Sriharsha Chintalapani
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26107/ --- (Updated Sept. 27, 2014, 2:13 a.m.) Review request for kafka. Bugs:

[jira] [Updated] (KAFKA-1577) Exception in ConnectionQuotas while shutting down

2014-09-26 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani updated KAFKA-1577: -- Attachment: KAFKA-1577_2014-09-26_19:13:05.patch Exception in ConnectionQuotas

[jira] [Commented] (KAFKA-1577) Exception in ConnectionQuotas while shutting down

2014-09-26 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150340#comment-14150340 ] Sriharsha Chintalapani commented on KAFKA-1577: --- Updated reviewboard

[jira] [Commented] (KAFKA-1618) Exception thrown when running console producer with no port number for the broker

2014-09-26 Thread BalajiSeshadri (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150350#comment-14150350 ] BalajiSeshadri commented on KAFKA-1618: --- [~nehanarkhede] Thanks a lot. Exception

Re: Review Request 26107: Patch for KAFKA-1577

2014-09-26 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26107/#review54764 --- Ship it! Ship It! - Joel Koshy On Sept. 27, 2014, 2:13 a.m.,

automatic topic creation question

2014-09-26 Thread Dave Peterson
I have a broker cluster configured with auto.create.topics.enable=true. Observing tcpdump output while using kafka-console-producer.sh to send a message with a previously nonexistent topic, it appears that the producer instructs the broker to create the topic by sending a metadata request for that