Re: [VOTE] 0.8.2.1 Candidate 2

2015-03-09 Thread Jun Rao
The following are the results of the votes. +1 binding = 3 votes +1 non-binding = 2 votes -1 = 0 votes 0 = 0 votes The vote passes. I will release artifacts to maven central, update the dist svn and download site. Will send out an announce after that. Thanks everyone that contributed to the

[jira] [Commented] (KAFKA-2010) unit tests sometimes are slow during shutdown

2015-03-09 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353190#comment-14353190 ] Jun Rao commented on KAFKA-2010: [~gwenshap], thanks for the info. Steve Miller also

[jira] [Created] (KAFKA-2011) Rebalance with auto.leader.rebalance.enable=false

2015-03-09 Thread K Zakee (JIRA)
K Zakee created KAFKA-2011: -- Summary: Rebalance with auto.leader.rebalance.enable=false Key: KAFKA-2011 URL: https://issues.apache.org/jira/browse/KAFKA-2011 Project: Kafka Issue Type: Bug

Re: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-09 Thread Todd Palino
First, a couple notes on this... 3 - I generally agree with the direction of not pre-optimizing. However, in this case I'm concerned about the calculation of the cost of doing plugins now vs. trying to refactor the code to do it later. It would seem to me that doing it up front will have less

[jira] [Commented] (KAFKA-1983) TestEndToEndLatency can be unreliable after hard kill

2015-03-09 Thread Grayson Chao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353237#comment-14353237 ] Grayson Chao commented on KAFKA-1983: - Mind if I work on this? I'm brand new to the

Re: [DISCUSS] KIP-11- Authorization design for kafka security

2015-03-09 Thread Don Bosco Durai
Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we

Re: [DISCUSS] KIP-11- Authorization design for kafka security

2015-03-09 Thread Parth Brahmbhatt
Hi Bosco, Thanks for taking the time to review. I will update the doc with a policy example and I will add CONNECT as an operation. For Admin APIs, I agree the best thing to do right now is just wait for KIP-4 to be submitted. I will update the doc to reflect the same. Thanks Parth On 3/9/15,

Can I be added as a contributor?

2015-03-09 Thread Grayson Chao
Hello Kafka devs, I'm working on the ops side of Kafka at LinkedIn (embedded SRE on the Kafka team) and would like to start familiarizing myself with the codebase with a view to eventually making substantial contributions. Could you please add me as a contributor to the Kafka JIRA so that I

Re: Review Request 31850: Patch for KAFKA-1660

2015-03-09 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31850/#review75731 ---

[jira] [Commented] (KAFKA-1987) Potential race condition in partition creation

2015-03-09 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353502#comment-14353502 ] Joel Koshy commented on KAFKA-1987: --- I actually think it would be worthwhile to improve

[jira] [Commented] (KAFKA-2011) Rebalance with auto.leader.rebalance.enable=false

2015-03-09 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353681#comment-14353681 ] Jiangjie Qin commented on KAFKA-2011: - [~kzakee] The log you pasted shows that there

[jira] [Commented] (KAFKA-2011) Rebalance with auto.leader.rebalance.enable=false

2015-03-09 Thread K Zakee (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353723#comment-14353723 ] K Zakee commented on KAFKA-2011: Can following error trigger the controller migration? I

Re: [DISCUSSION] KIP-15 close(timeout) for producer

2015-03-09 Thread Jiangjie Qin
Hi Jay, I have updated the KIP as you suggested. -Jiangjie (Becket) Qin On 3/8/15, 9:45 AM, Jay Kreps jay.kr...@gmail.com wrote: Hey Jiangjie, Can you capture the full motivation and use cases for the feature? This mentions your interest in having a way of aborting from inside the Callback.

adding replicas to existing topic partitions - KAFKA-1313

2015-03-09 Thread Geoffrey Anderson
Hi dev list, I have a few questions regarding KAFKA-1313 ( https://issues.apache.org/jira/browse/KAFKA-1313) It is currently possible albeit in a kindof hacky way to increase replication factor on a topic by using the partition reassignment tool with a hand-crafted reassignment plan. The

[jira] [Created] (KAFKA-2012) Broker should automatically handle corrupt index files

2015-03-09 Thread Todd Palino (JIRA)
Todd Palino created KAFKA-2012: -- Summary: Broker should automatically handle corrupt index files Key: KAFKA-2012 URL: https://issues.apache.org/jira/browse/KAFKA-2012 Project: Kafka Issue Type:

[jira] [Commented] (KAFKA-1501) transient unit tests failures due to port already in use

2015-03-09 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353363#comment-14353363 ] Ewen Cheslack-Postava commented on KAFKA-1501: -- Updated reviewboard

Re: Review Request 31850: Patch for KAFKA-1660

2015-03-09 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31850/ --- (Updated March 9, 2015, 7:56 p.m.) Review request for kafka. Bugs:

Re: Review Request 31806: Patch for KAFKA-1501

2015-03-09 Thread Ewen Cheslack-Postava
On March 9, 2015, 3:26 a.m., Eric Olander wrote: core/src/test/scala/integration/kafka/api/ProducerFailureHandlingTest.scala, line 274 https://reviews.apache.org/r/31806/diff/1/?file=887521#file887521line274 This is confusing - this inner loop doesn't use i so I'm not sure why

Re: Can I be added as a contributor?

2015-03-09 Thread Guozhang Wang
Added grayson.c...@gmail.com to the list. On Mon, Mar 9, 2015 at 10:41 AM, Grayson Chao gc...@linkedin.com.invalid wrote: Hello Kafka devs, I'm working on the ops side of Kafka at LinkedIn (embedded SRE on the Kafka team) and would like to start familiarizing myself with the codebase with a

[jira] [Updated] (KAFKA-1983) TestEndToEndLatency can be unreliable after hard kill

2015-03-09 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-1983: Assignee: Grayson Chao TestEndToEndLatency can be unreliable after hard kill

[jira] [Commented] (KAFKA-2011) Rebalance with auto.leader.rebalance.enable=false

2015-03-09 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353362#comment-14353362 ] Jiangjie Qin commented on KAFKA-2011: - Hmm, it looks you have controller migration.

Re: Review Request 31850: Patch for KAFKA-1660

2015-03-09 Thread Jiangjie Qin
On March 9, 2015, 6:37 p.m., Guozhang Wang wrote: clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java, line 565 https://reviews.apache.org/r/31850/diff/2/?file=86#file86line565 Could metrics.close() be called simultaneously? I think so. It is

Re: Review Request 31806: Patch for KAFKA-1501

2015-03-09 Thread Ewen Cheslack-Postava
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31806/ --- (Updated March 9, 2015, 6:41 p.m.) Review request for kafka. Bugs:

[jira] [Updated] (KAFKA-1501) transient unit tests failures due to port already in use

2015-03-09 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-1501: - Attachment: KAFKA-1501_2015-03-09_11:41:07.patch transient unit tests failures

[jira] [Commented] (KAFKA-1983) TestEndToEndLatency can be unreliable after hard kill

2015-03-09 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353366#comment-14353366 ] Jiangjie Qin commented on KAFKA-1983: - Welcome, [~gchao]. Just assigned the ticket to

[jira] [Commented] (KAFKA-2011) Rebalance with auto.leader.rebalance.enable=false

2015-03-09 Thread K Zakee (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353465#comment-14353465 ] K Zakee commented on KAFKA-2011: Nothing per se happened to brokers. No restarts or hung.

Re: Review Request 31850: Patch for KAFKA-1660

2015-03-09 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31850/#review75742 ---

[jira] [Updated] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.

2015-03-09 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-1660: Attachment: KAFKA-1660_2015-03-09_12:56:39.patch Ability to call close() with a timeout on the

[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.

2015-03-09 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353478#comment-14353478 ] Jiangjie Qin commented on KAFKA-1660: - Updated reviewboard

Re: Review Request 31706: Patch for KAFKA-1997

2015-03-09 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31706/#review75785 --- Patch does not apply. Can you rebase?

Re: Review Request 30809: Patch for KAFKA-1888

2015-03-09 Thread Abhishek Nigam
On Feb. 18, 2015, 12:06 a.m., Mayuresh Gharat wrote: core/src/main/scala/kafka/tools/ContinuousValidationTest.java, line 400 https://reviews.apache.org/r/30809/diff/1/?file=859055#file859055line400 The common format of commenting is : // this is a comment

Re: Review Request 31706: Patch for KAFKA-1997

2015-03-09 Thread Joel Koshy
On March 6, 2015, 12:14 a.m., Joel Koshy wrote: core/src/main/scala/kafka/consumer/ZookeeperConsumerConnector.scala, line 729 https://reviews.apache.org/r/31706/diff/4/?file=885022#file885022line729 partitionOwnerships - topicPartitionAssignment Jiangjie Qin wrote: I

Re: Review Request 30809: Patch for KAFKA-1888

2015-03-09 Thread Abhishek Nigam
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30809/ --- (Updated March 9, 2015, 11:55 p.m.) Review request for kafka. Bugs:

Re: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-09 Thread Jay Kreps
Hey Todd, Nice points, let me try to respond: Plugins Yeah let me explain what I mean about plugins. The contracts that matter for us are public contracts, i.e. the protocol, the binary format, stuff in zk, and the various plug-in apis we expose. Adding an internal interface later will not be

[jira] [Commented] (KAFKA-2003) Add upgrade tests

2015-03-09 Thread Abhishek Nigam (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353922#comment-14353922 ] Abhishek Nigam commented on KAFKA-2003: --- Hi Ashish/Gwen, Can you review KAFKA-1888

[jira] [Commented] (KAFKA-2010) unit tests sometimes are slow during shutdown

2015-03-09 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353958#comment-14353958 ] Guozhang Wang commented on KAFKA-2010: -- Jun, how frequently will this happen on your

[jira] [Commented] (KAFKA-2011) Rebalance with auto.leader.rebalance.enable=false

2015-03-09 Thread K Zakee (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353848#comment-14353848 ] K Zakee commented on KAFKA-2011: Attached the controller logs for your reference.

[jira] [Updated] (KAFKA-2011) Rebalance with auto.leader.rebalance.enable=false

2015-03-09 Thread K Zakee (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] K Zakee updated KAFKA-2011: --- Attachment: controller-logs-2.zip controller-logs-1.zip Rebalance with

Re: Review Request 30809: Patch for KAFKA-1888

2015-03-09 Thread Abhishek Nigam
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30809/#review73061 --- core/src/main/scala/kafka/tools/ContinuousValidationTest.java

Re: Review Request 30809: Patch for KAFKA-1888

2015-03-09 Thread Abhishek Nigam
On Feb. 18, 2015, 12:06 a.m., Mayuresh Gharat wrote: core/src/main/scala/kafka/tools/ContinuousValidationTest.java, line 168 https://reviews.apache.org/r/30809/diff/1/?file=859055#file859055line168 same here can we use isInterrupted()?

[jira] [Commented] (KAFKA-1461) Replica fetcher thread does not implement any back-off behavior

2015-03-09 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354272#comment-14354272 ] Jun Rao commented on KAFKA-1461: [~sriharsha], thanks. Your understanding is correct. We

Re: adding replicas to existing topic partitions - KAFKA-1313

2015-03-09 Thread Ewen Cheslack-Postava
Geoff, First, if you haven't already, take a look at https://cwiki.apache.org/confluence/display/KAFKA/KIP-6+-+New+reassignment+partition+logic+for+rebalancing and the associated JIRA. My opinion is that we shouldn't do anything to the existing partitions -- it would probably violate the

[jira] [Commented] (KAFKA-1990) Add unlimited time-based log retention

2015-03-09 Thread Jeff Holoman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354325#comment-14354325 ] Jeff Holoman commented on KAFKA-1990: - Updated reviewboard

Re: Can I be added as a contributor?

2015-03-09 Thread Joe Stein
Grant, I added you. Grayson and Grant, you should both also please setup Confluence accounts and we can grant perms to Confluence also too for your username. ~ Joe Stein - - - - - - - - - - - - - - - - - http://www.stealth.ly - - - - - - - - - - - - - - - - - On Tue, Mar 10, 2015 at 12:54

[jira] [Commented] (KAFKA-1930) Move server over to new metrics library

2015-03-09 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354269#comment-14354269 ] Gwen Shapira commented on KAFKA-1930: - Also needed: metrics report via Servlets (also

[jira] [Commented] (KAFKA-1930) Move server over to new metrics library

2015-03-09 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354303#comment-14354303 ] Gwen Shapira commented on KAFKA-1930: - Actually, can someone who was on the original

[jira] [Commented] (KAFKA-1930) Move server over to new metrics library

2015-03-09 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354338#comment-14354338 ] Gwen Shapira commented on KAFKA-1930: - Never mind :) [~joestein] kindly pointed me at

Re: Review Request 31590: Patch for KAFKA-1990

2015-03-09 Thread Jeff Holoman
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31590/ --- (Updated March 10, 2015, 4:58 a.m.) Review request for kafka and Gwen Shapira.

Re: Review Request 31590: Patch for KAFKA-1990

2015-03-09 Thread Jeff Holoman
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31590/ --- (Updated March 10, 2015, 4:55 a.m.) Review request for kafka and Gwen Shapira.

Re: Can I be added as a contributor?

2015-03-09 Thread Grant Henke
I am also starting to work with the Kafka codebase with plans to contribute more significantly in the near future. Could I also be added to the contributor list so that I can assign myself tickets? Thank you, Grant On Mon, Mar 9, 2015 at 1:39 PM, Guozhang Wang wangg...@gmail.com wrote: Added

[jira] [Commented] (KAFKA-1930) Move server over to new metrics library

2015-03-09 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354376#comment-14354376 ] Jay Kreps commented on KAFKA-1930: -- Hey [~gwenshap] this is wheel reinvention, no

[jira] [Commented] (KAFKA-1461) Replica fetcher thread does not implement any back-off behavior

2015-03-09 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354139#comment-14354139 ] Sriharsha Chintalapani commented on KAFKA-1461: --- [~junrao] I'll work

[jira] [Commented] (KAFKA-2010) unit tests sometimes are slow during shutdown

2015-03-09 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354072#comment-14354072 ] Jun Rao commented on KAFKA-2010: [~guozhang], this seems to happen frequently to me on

[jira] [Commented] (KAFKA-1461) Replica fetcher thread does not implement any back-off behavior

2015-03-09 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354081#comment-14354081 ] Jun Rao commented on KAFKA-1461: [~sriharsha], since this affects the unit test

Re: Review Request 31706: Patch for KAFKA-1997

2015-03-09 Thread Jiangjie Qin
On March 9, 2015, 10:34 p.m., Joel Koshy wrote: core/src/main/scala/kafka/tools/MirrorMaker.scala, line 668 https://reviews.apache.org/r/31706/diff/5/?file=887054#file887054line668 Failed to send message to ... Also, it is probably better to _not_ log the number of

[jira] [Updated] (KAFKA-1997) Refactor Mirror Maker

2015-03-09 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-1997: Attachment: KAFKA-1997_2015-03-09_18:55:54.patch Refactor Mirror Maker -

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-03-09 Thread Aditya Auradkar
Thanks for the comments Jay and Todd. Firstly, I'd like to address some of Jay's points. 1. You are right that we don't need to disable quotas on a per-client basis. 2. Configuration management: I must admit, I haven't read Joe's proposal in detail. My thinking was that we can keep

[jira] [Commented] (KAFKA-1461) Replica fetcher thread does not implement any back-off behavior

2015-03-09 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354068#comment-14354068 ] Jun Rao commented on KAFKA-1461: [~sriharsha], thanks for the patch. Managing the backoff

Re: Review Request 31706: Patch for KAFKA-1997

2015-03-09 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31706/ --- (Updated March 10, 2015, 1:55 a.m.) Review request for kafka. Bugs:

[jira] [Commented] (KAFKA-1997) Refactor Mirror Maker

2015-03-09 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354104#comment-14354104 ] Jiangjie Qin commented on KAFKA-1997: - Updated reviewboard