[jira] [Updated] (KAFKA-2178) Loss of highwatermarks on incorrect cluster shutdown/restart

2015-05-07 Thread Alexey Ozeritskiy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Ozeritskiy updated KAFKA-2178: - Status: Patch Available (was: Open) Loss of highwatermarks on incorrect cluster

[jira] [Created] (KAFKA-2180) topics never create on brokers though it succeeds in tool and is in zookeeper

2015-05-07 Thread Joe Stein (JIRA)
Joe Stein created KAFKA-2180: Summary: topics never create on brokers though it succeeds in tool and is in zookeeper Key: KAFKA-2180 URL: https://issues.apache.org/jira/browse/KAFKA-2180 Project: Kafka

Re: [DISCUSS] KIP-23 - Add JSON/CSV output and looping options to ConsumerGroupCommand

2015-05-07 Thread Ashish Singh
Had to change the title of the page and that surprisingly changed the link as well. KIP-23 is now available at here https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=56852556. On Thu, May 7, 2015 at 11:34 AM, Ashish Singh asi...@cloudera.com wrote: Hi Guys, I just added a KIP,

[jira] [Updated] (KAFKA-2178) Loss of highwatermarks on incorrect cluster shutdown/restart

2015-05-07 Thread Alexey Ozeritskiy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Ozeritskiy updated KAFKA-2178: - Attachment: KAFKA-2178.patch Loss of highwatermarks on incorrect cluster

[jira] [Created] (KAFKA-2178) Loss of highwatermarks on incorrect cluster shutdown/restart

2015-05-07 Thread Alexey Ozeritskiy (JIRA)
Alexey Ozeritskiy created KAFKA-2178: Summary: Loss of highwatermarks on incorrect cluster shutdown/restart Key: KAFKA-2178 URL: https://issues.apache.org/jira/browse/KAFKA-2178 Project: Kafka

[DISCUSS] KIP-23 - Add JSON/CSV output and looping options to ConsumerGroupCommand

2015-05-07 Thread Ashish Singh
Hi Guys, I just added a KIP, KIP-23 - Add JSON/CSV output and looping options to ConsumerGroupCommand https://cwiki.apache.org/confluence/display/KAFKA/KIP-23, for KAFKA-313 https://issues.apache.org/jira/browse/KAFKA-313. The changes made as part of the JIRA can be found here

[jira] [Commented] (KAFKA-313) Add JSON/CSV output and looping options to ConsumerGroupCommand

2015-05-07 Thread Ashish K Singh (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533166#comment-14533166 ] Ashish K Singh commented on KAFKA-313: -- [~nehanarkhede] thanks for the review! Makes

Re: Need Access to Wiki Page To Create Page for Discussion

2015-05-07 Thread Bhavesh Mistry
Hi Guozhang, Here is link to account https://cwiki.apache.org/confluence/users/viewuserprofile.action?username=bmis13 I just created the account wiki. I was under the impression that jira account is wiki account. Thanks in advance for help ! Thanks, Bhavesh On Wed, May 6, 2015 at 8:33 PM,

[jira] [Created] (KAFKA-2179) no graceful nor fast way to shutdown every broker without killing them

2015-05-07 Thread Joe Stein (JIRA)
Joe Stein created KAFKA-2179: Summary: no graceful nor fast way to shutdown every broker without killing them Key: KAFKA-2179 URL: https://issues.apache.org/jira/browse/KAFKA-2179 Project: Kafka

[jira] [Created] (KAFKA-2177) Modify o.a.k.clients.NetworkClient to reuse it in controller/replica fetchers.

2015-05-07 Thread Jiangjie Qin (JIRA)
Jiangjie Qin created KAFKA-2177: --- Summary: Modify o.a.k.clients.NetworkClient to reuse it in controller/replica fetchers. Key: KAFKA-2177 URL: https://issues.apache.org/jira/browse/KAFKA-2177 Project:

[jira] [Commented] (KAFKA-2169) Upgrade to zkclient-0.5

2015-05-07 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533309#comment-14533309 ] ASF GitHub Bot commented on KAFKA-2169: --- GitHub user Parth-Brahmbhatt opened a pull

Why we call callback.onCompletion in send()?

2015-05-07 Thread Jiangjie Qin
Hi, I just notice that in new producer we actually call callback.onCompleteion() in the catch clause of send(). This seems breaking the callback execution order guarantee we provided. Any reason why we do this instead of just throw an exception? Jiangjie (Becket) Qin

[GitHub] kafka pull request: KAFKA-2169: Moving to zkClient 0.5 release.

2015-05-07 Thread Parth-Brahmbhatt
GitHub user Parth-Brahmbhatt opened a pull request: https://github.com/apache/kafka/pull/61 KAFKA-2169: Moving to zkClient 0.5 release. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Parth-Brahmbhatt/kafka KAFKA-2169

[jira] [Created] (KAFKA-2181) online doc issues

2015-05-07 Thread Jason Rosenberg (JIRA)
Jason Rosenberg created KAFKA-2181: -- Summary: online doc issues Key: KAFKA-2181 URL: https://issues.apache.org/jira/browse/KAFKA-2181 Project: Kafka Issue Type: Bug Reporter:

Re: Review Request 33088: add heartbeat to coordinator

2015-05-07 Thread Onur Karaman
On May 7, 2015, 10:07 p.m., Guozhang Wang wrote: I think we can check this in if you agree with the following minor comments (I can do them while committing), and then move on to the rebalance logic / leftover works. Guozhang Wang wrote: Also could you rebase since currently

RE: [DISCUSS] KIP-21 Configuration Management

2015-05-07 Thread Aditya Auradkar
Theoretically, using just the broker id and zk connect string it should be possible for the broker to read all configs from Zookeeper. Like Ashish said, we should probably take a look and make sure. Additionally, we've spoken about making config changes only through a broker API. However, we

Re: Review Request 33620: Patch for KAFKA-1690

2015-05-07 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33620/#review82953 --- Thanks for the patch. Just a few quick comments.

[jira] [Commented] (KAFKA-1690) new java producer needs ssl support as a client

2015-05-07 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533747#comment-14533747 ] Jun Rao commented on KAFKA-1690: Joel mentioned the use case that returning just the peer

Build failed in Jenkins: KafkaPreCommit #96

2015-05-07 Thread Apache Jenkins Server
See https://builds.apache.org/job/KafkaPreCommit/96/changes Changes: [wangguoz] KAFKA-1884; Add logging upon metadata response errors; reviewed by Guozhang Wang -- [...truncated 411 lines...] org.apache.kafka.common.record.RecordTest testEquality[58]

[jira] [Commented] (KAFKA-2172) Round-robin partition assignment strategy too restrictive

2015-05-07 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533651#comment-14533651 ] Joel Koshy commented on KAFKA-2172: --- The restriction really has to do with simplicity in

[jira] [Updated] (KAFKA-1884) Print metadata response errors

2015-05-07 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1884: - Resolution: Fixed Status: Resolved (was: Patch Available) Print metadata response

Re: Review Request 33620: Patch for KAFKA-1690

2015-05-07 Thread Sriharsha Chintalapani
On May 8, 2015, 1:41 a.m., Jun Rao wrote: clients/src/main/java/org/apache/kafka/common/network/DefaultAuthenticator.java, lines 37-39 https://reviews.apache.org/r/33620/diff/1/?file=943814#file943814line37 Do we want to return Principal or Subject? It seems that ssl only gives

[jira] [Updated] (KAFKA-2176) DefaultPartitioner doesn't perform consistent hashing based on

2015-05-07 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Maravić updated KAFKA-2176: Status: Patch Available (was: Open) DefaultPartitioner doesn't perform consistent hashing based

[jira] [Updated] (KAFKA-2176) DefaultPartitioner doesn't perform consistent hashing based on

2015-05-07 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Maravić updated KAFKA-2176: Status: Patch Available (was: Open) DefaultPartitioner doesn't perform consistent hashing based

[jira] [Created] (KAFKA-2176) DefaultPartitioner doesn't perform consistent hashing based on

2015-05-07 Thread JIRA
Igor Maravić created KAFKA-2176: --- Summary: DefaultPartitioner doesn't perform consistent hashing based on Key: KAFKA-2176 URL: https://issues.apache.org/jira/browse/KAFKA-2176 Project: Kafka

[jira] [Commented] (KAFKA-1690) new java producer needs ssl support as a client

2015-05-07 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533756#comment-14533756 ] Sriharsha Chintalapani commented on KAFKA-1690: --- [~junrao] [~jjkoshy] I do

Re: Review Request 33088: add heartbeat to coordinator

2015-05-07 Thread Onur Karaman
On May 7, 2015, 10:07 p.m., Guozhang Wang wrote: core/src/main/scala/kafka/coordinator/CoordinatorMetadata.scala, line 53 https://reviews.apache.org/r/33088/diff/10/?file=950430#file950430line53 Add @nonthreadsafe for this function. It would be awkward to have the class claim it's

[jira] [Commented] (KAFKA-2121) prevent potential resource leak in KafkaProducer and KafkaConsumer

2015-05-07 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533822#comment-14533822 ] Guozhang Wang commented on KAFKA-2121: -- Committed the followup patch to trunk.

Re: Review Request 33620: Patch for KAFKA-1690

2015-05-07 Thread Sriharsha Chintalapani
On May 8, 2015, 1:41 a.m., Jun Rao wrote: clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java, line 288 https://reviews.apache.org/r/33620/diff/1/?file=943817#file943817line288 Ssl can trigger rehandshake. How is the rehandshake triggered? Do we need

Build failed in Jenkins: Kafka-trunk #485

2015-05-07 Thread Apache Jenkins Server
See https://builds.apache.org/job/Kafka-trunk/485/changes Changes: [wangguoz] KAFKA-2121; Fix Closeable backward-compatibility; reviewed by Guozhang Wang -- Started by an SCM change Building remotely on ubuntu3 (Ubuntu ubuntu legacy-ubuntu) in workspace

Jenkins build is back to normal : Kafka-trunk #486

2015-05-07 Thread Apache Jenkins Server
See https://builds.apache.org/job/Kafka-trunk/486/changes

Build failed in Jenkins: KafkaPreCommit #97

2015-05-07 Thread Apache Jenkins Server
See https://builds.apache.org/job/KafkaPreCommit/97/changes Changes: [wangguoz] KAFKA-2121; Fix Closeable backward-compatibility; reviewed by Guozhang Wang -- Started by an SCM change Building remotely on ubuntu3 (Ubuntu ubuntu legacy-ubuntu) in

Build failed in Jenkins: KafkaPreCommit #98

2015-05-07 Thread Apache Jenkins Server
See https://builds.apache.org/job/KafkaPreCommit/98/changes Changes: [wangguoz] KAFKA-2121; add missing file -- [...truncated 1268 lines...] kafka.consumer.TopicFilterTest testWildcardTopicCountGetTopicCountMapEscapeJson PASSED

[jira] [Updated] (KAFKA-2121) prevent potential resource leak in KafkaProducer and KafkaConsumer

2015-05-07 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2121: - Resolution: Fixed Status: Resolved (was: Patch Available) prevent potential resource

[jira] [Commented] (KAFKA-1387) Kafka getting stuck creating ephemeral node it has already created when two zookeeper sessions are established in a very short period of time

2015-05-07 Thread Abhishek Nigam (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533402#comment-14533402 ] Abhishek Nigam commented on KAFKA-1387: --- I have seen the ephemeral node issue before

Re: [VOTE] KIP-4 Admin Commands / Phase-1

2015-05-07 Thread Jun Rao
+1 on the vote. Thanks, Jun On Thu, May 7, 2015 at 8:29 AM, Andrii Biletskyi andrii.bilets...@stealth.ly wrote: Jun, 1. My bad, updated description, but forgot the schema itself. Fixed. 2. Okay, changed that. 3. Sure, we'll evolve TMR again once needed. Thanks, Andrii Biletskyi On

Re: Need Access to Wiki Page To Create Page for Discussion

2015-05-07 Thread Guozhang Wang
Done. Cheers, Guozhang On Thu, May 7, 2015 at 12:00 PM, Bhavesh Mistry mistry.p.bhav...@gmail.com wrote: Hi Guozhang, Here is link to account https://cwiki.apache.org/confluence/users/viewuserprofile.action?username=bmis13 I just created the account wiki. I was under the impression that

Re: Review Request 33088: add heartbeat to coordinator

2015-05-07 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33088/#review82921 --- Ship it! I think we can check this in if you agree with the

[jira] [Commented] (KAFKA-1884) Print metadata response errors

2015-05-07 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533539#comment-14533539 ] Guozhang Wang commented on KAFKA-1884: -- [~omkreddy] Sorry for the late reply,

Environment variables for network config

2015-05-07 Thread Lukas Steiblys
Hello Kafka developers, We have built a prototype analytics processing pipeline using Kafka and Samza and are working on making it production ready. We are using Docker to containerize Kafka service and one problem we are having is specifying configuration options in properties files,

Re: Review Request 33731: Second Attempt to Fix KAFKA-2160

2015-05-07 Thread Onur Karaman
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33731/#review82911 --- I'm guessing the answer is yes, but is it too much of a performance

Re: Review Request 31627: Patch for KAFKA-1884

2015-05-07 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31627/#review82926 --- Ship it!

Re: Review Request 33088: add heartbeat to coordinator

2015-05-07 Thread Guozhang Wang
On May 7, 2015, 10:07 p.m., Guozhang Wang wrote: I think we can check this in if you agree with the following minor comments (I can do them while committing), and then move on to the rebalance logic / leftover works. Also could you rebase since currently patch does not apply cleanly on

Re: [DISCUSSION] Reuse o.a.k.clients.NetworkClient in controller.

2015-05-07 Thread Jay Kreps
Hey guys, I haven't thought this all the way through, but I think having metadata in the NetworkClient actually does make sense. After all it is the case that all requests are directed to Kafka nodes so this is a higher level of abstraction to work at. I think the feature you are looking for is

Re: [DISCUSS] KIP-21 Configuration Management

2015-05-07 Thread Jun Rao
Ashish, 3. This is true. However, using files has the same problem. You can't store the location of the file in the file itself. The location of the file has to be passed out of band into Kafka. Thanks, Jun On Wed, May 6, 2015 at 6:34 PM, Ashish Singh asi...@cloudera.com wrote: Hey Jun,

Re: [VOTE] KIP-4 Admin Commands / Phase-1

2015-05-07 Thread Jun Rao
Hi, Andrii, A few more comments. 1. We need to remove isr and replicaLags from TMR, right? 2. For TMR v1, we decided to keep the same auto topic creation logic as v0 initially. We can drop the auto topic creation logic later after the producer client starts using createTopicRequest. 3. For the

Re: [VOTE] KIP-4 Admin Commands / Phase-1

2015-05-07 Thread Andrii Biletskyi
Jun, 1. My bad, updated description, but forgot the schema itself. Fixed. 2. Okay, changed that. 3. Sure, we'll evolve TMR again once needed. Thanks, Andrii Biletskyi On Thu, May 7, 2015 at 6:16 PM, Jun Rao j...@confluent.io wrote: Hi, Andrii, A few more comments. 1. We need to remove

[jira] [Updated] (KAFKA-2176) DefaultPartitioner doesn't perform consistent hashing based on

2015-05-07 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Maravić updated KAFKA-2176: Status: Open (was: Patch Available) DefaultPartitioner doesn't perform consistent hashing based

[jira] [Commented] (KAFKA-2176) DefaultPartitioner doesn't perform consistent hashing based on

2015-05-07 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14532514#comment-14532514 ] Igor Maravić commented on KAFKA-2176: - Created reviewboard

Review Request 33939: Patch for KAFKA-2176

2015-05-07 Thread Igor Maravi?
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33939/ --- Review request for kafka. Bugs: KAFKA-2176

[jira] [Updated] (KAFKA-2176) DefaultPartitioner doesn't perform consistent hashing based on

2015-05-07 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Maravić updated KAFKA-2176: Attachment: KAFKA-2176.patch DefaultPartitioner doesn't perform consistent hashing based on