[jira] [Commented] (KAFKA-2084) byte rate metrics per client ID (producer and consumer)

2015-08-10 Thread Aditya A Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681248#comment-14681248 ] Aditya A Auradkar commented on KAFKA-2084: -- Updated reviewboard

Re: Review Request 33049: Patch for KAFKA-2084

2015-08-10 Thread Aditya Auradkar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33049/ --- (Updated Aug. 11, 2015, 4:58 a.m.) Review request for kafka, Joel Koshy and

Re: Review Request 33049: Patch for KAFKA-2084

2015-08-10 Thread Aditya Auradkar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33049/ --- (Updated Aug. 11, 2015, 4:57 a.m.) Review request for kafka, Joel Koshy and

Re: Review Request 33378: Patch for KAFKA-2136

2015-08-10 Thread Aditya Auradkar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33378/#review94876 --- Jun/Joel - Thanks for the comments. I'd like to address these all

[jira] [Updated] (KAFKA-2084) byte rate metrics per client ID (producer and consumer)

2015-08-10 Thread Aditya A Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya A Auradkar updated KAFKA-2084: - Attachment: KAFKA-2084_2015-08-10_21:57:48.patch byte rate metrics per client ID

[jira] [Commented] (KAFKA-2071) Replace Produce Request/Response with their org.apache.kafka.common.requests equivalents

2015-08-10 Thread David Jacot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681274#comment-14681274 ] David Jacot commented on KAFKA-2071: I just updated the PR. Now, ProducerRequest and

[jira] [Updated] (KAFKA-2071) Replace Produce Request/Response with their org.apache.kafka.common.requests equivalents

2015-08-10 Thread David Jacot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot updated KAFKA-2071: --- Status: Patch Available (was: In Progress) Replace Produce Request/Response with their

[jira] [Commented] (KAFKA-2196) remove roundrobin identical topic constraint in consumer coordinator

2015-08-10 Thread Andrew Olson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680204#comment-14680204 ] Andrew Olson commented on KAFKA-2196: - Doesn't this [1] code also need to be updated?

Re: Review Request 34766: Patch for KAFKA-2229

2015-08-10 Thread Ismael Juma
On Aug. 10, 2015, 3:47 p.m., Grant Henke wrote: My apologies for not looking at this sooner, or suggesting this sooner. Given that this code change and scope is fairly large, would it be too much work to break out the patches reviews by each new protocol message? Then reviews can be

Re: [DISCUSSION] KIP-29 - Add an IsrPropagateIntervalMs config to KafkaConfig

2015-08-10 Thread Ashish Singh
Hey Guys, Looks like Jun and Jiangjie have covered the motivation behind the config. To me one would want to set ISR propagation delay to a high number primarily during rolling upgrade. I think the delay one would want to have in ISR propagation is proportional to the cluster size. However,

Re: [DISCUSSION] KIP-29 - Add an IsrPropagateIntervalMs config to KafkaConfig

2015-08-10 Thread Jay Kreps
I guess the question, which I think is what Gwen was getting at, is if, rather than making this configurable, it might be possible to make this just work reliably and with the lowest possible latency in some automatic fashion? I raised group commit because that is a way to automatically batch

Re: Review Request 34766: Patch for KAFKA-2229

2015-08-10 Thread Grant Henke
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34766/#review94749 --- My apologies for not looking at this sooner, or suggesting this

Re: Review Request 34766: Patch for KAFKA-2229

2015-08-10 Thread Andrii Biletskyi
On Aug. 10, 2015, 3:47 p.m., Grant Henke wrote: My apologies for not looking at this sooner, or suggesting this sooner. Given that this code change and scope is fairly large, would it be too much work to break out the patches reviews by each new protocol message? Then reviews can be

Re: Review Request 34766: Patch for KAFKA-2229

2015-08-10 Thread Grant Henke
On Aug. 10, 2015, 3:47 p.m., Grant Henke wrote: My apologies for not looking at this sooner, or suggesting this sooner. Given that this code change and scope is fairly large, would it be too much work to break out the patches reviews by each new protocol message? Then reviews can be

Re: [jira] [Commented] (KAFKA-1690) new java producer needs ssl support as a client

2015-08-10 Thread Sriharsha Chintalapani
Thanks for testing out Rajini. I did ran that test in a loop and it never hanged for me. I am hoping you are using the latest patch since the data left over issue is addressed in latest patch. Also if thats an issue SSLConsumerTest and SSLProducerTest will hang too. Did you notice those are

Re: Review Request 34766: Patch for KAFKA-2229

2015-08-10 Thread Andrii Biletskyi
On Aug. 10, 2015, 3:47 p.m., Grant Henke wrote: My apologies for not looking at this sooner, or suggesting this sooner. Given that this code change and scope is fairly large, would it be too much work to break out the patches reviews by each new protocol message? Then reviews can be

[jira] [Assigned] (KAFKA-1811) ensuring registered broker host:port is unique

2015-08-10 Thread Edward Ribeiro (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Ribeiro reassigned KAFKA-1811: - Assignee: Edward Ribeiro ensuring registered broker host:port is unique

Re: Review Request 34766: Patch for KAFKA-2229

2015-08-10 Thread Grant Henke
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34766/#review94759 --- core/src/main/scala/kafka/server/TopicCommandHelper.scala (line

Re: [DISCUSSION] KIP-29 - Add an IsrPropagateIntervalMs config to KafkaConfig

2015-08-10 Thread Ashish Singh
Not having to deal with configs sounds ideal. I liked the idea of having automatic batching, however that requires broker-to-broker comm support. At some point, I think it will be better to have direct broker - controller communication, rather that using ZK for the purpose. Few more approaches.

[jira] [Commented] (KAFKA-2084) byte rate metrics per client ID (producer and consumer)

2015-08-10 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680409#comment-14680409 ] Aditya Auradkar commented on KAFKA-2084: [~junrao][~jjkoshy] I think I've

[jira] [Commented] (KAFKA-1695) Authenticate connection to Zookeeper

2015-08-10 Thread Parth Brahmbhatt (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680417#comment-14680417 ] Parth Brahmbhatt commented on KAFKA-1695: - [~ijuma] Don't have a date, have sent

[jira] [Commented] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.

2015-08-10 Thread Parth Brahmbhatt (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680464#comment-14680464 ] Parth Brahmbhatt commented on KAFKA-2210: - [~ijuma] Sorry got busy with some other

[GitHub] kafka pull request: KAFKA-2134: fix replica offset truncate to beg...

2015-08-10 Thread becketqin
Github user becketqin closed the pull request at: https://github.com/apache/kafka/pull/104 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Created] (KAFKA-2417) Ducktape tests for SSL/TLS

2015-08-10 Thread Ismael Juma (JIRA)
Ismael Juma created KAFKA-2417: -- Summary: Ducktape tests for SSL/TLS Key: KAFKA-2417 URL: https://issues.apache.org/jira/browse/KAFKA-2417 Project: Kafka Issue Type: Sub-task

[jira] [Commented] (KAFKA-2143) Replicas get ahead of leader and fail

2015-08-10 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680467#comment-14680467 ] ASF GitHub Bot commented on KAFKA-2143: --- GitHub user becketqin opened a pull

[jira] [Commented] (KAFKA-1695) Authenticate connection to Zookeeper

2015-08-10 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680466#comment-14680466 ] Ismael Juma commented on KAFKA-1695: [~parth.brahmbhatt], thanks. Authenticate

[GitHub] kafka pull request: KAFKA-2143: fix replica offset truncate to beg...

2015-08-10 Thread becketqin
GitHub user becketqin opened a pull request: https://github.com/apache/kafka/pull/129 KAFKA-2143: fix replica offset truncate to beginning during leader migration. You can merge this pull request into a Git repository by running: $ git pull https://github.com/becketqin/kafka

[jira] [Commented] (KAFKA-2134) Producer blocked on metric publish

2015-08-10 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680531#comment-14680531 ] ASF GitHub Bot commented on KAFKA-2134: --- Github user becketqin closed the pull

Re: Review Request 34766: Patch for KAFKA-2229

2015-08-10 Thread Grant Henke
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34766/#review94756 --- clients/src/main/java/org/apache/kafka/common/protocol/Errors.java

[GitHub] kafka pull request: KAFKA-1893: Allow regex subscriptions in the n...

2015-08-10 Thread SinghAsDev
GitHub user SinghAsDev opened a pull request: https://github.com/apache/kafka/pull/128 KAFKA-1893: Allow regex subscriptions in the new consumer You can merge this pull request into a Git repository by running: $ git pull https://github.com/SinghAsDev/kafka KAFKA-1893

[jira] [Commented] (KAFKA-1893) Allow regex subscriptions in the new consumer

2015-08-10 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680461#comment-14680461 ] ASF GitHub Bot commented on KAFKA-1893: --- GitHub user SinghAsDev opened a pull

Re: Review Request 34766: Patch for KAFKA-2229

2015-08-10 Thread Andrii Biletskyi
On Aug. 10, 2015, 5:01 p.m., Grant Henke wrote: core/src/main/scala/kafka/server/TopicCommandHelper.scala, line 35 https://reviews.apache.org/r/34766/diff/2/?file=995963#file995963line35 A lot of this code/functionality exists in kafka.admin.TopicCommand. Is this duplicating a

Typo on documentation

2015-08-10 Thread Edward Ribeiro
I have just seen the typo below at http://kafka.apache.org/documentation.html . It's supposed to be JMX instead of JMZ, right? []'s Eddie

[jira] [Updated] (KAFKA-313) Add JSON/CSV output and looping options to ConsumerGroupCommand

2015-08-10 Thread Ashish K Singh (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish K Singh updated KAFKA-313: - Attachment: KAFKA-313_2015-08-10_12:58:38.patch Add JSON/CSV output and looping options to

[jira] [Commented] (KAFKA-313) Add JSON/CSV output and looping options to ConsumerGroupCommand

2015-08-10 Thread Ashish K Singh (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680671#comment-14680671 ] Ashish K Singh commented on KAFKA-313: -- Updated reviewboard

Re: Typo on documentation

2015-08-10 Thread Gwen Shapira
yeppers. JIRA and patch? On Mon, Aug 10, 2015 at 12:36 PM, Edward Ribeiro edward.ribe...@gmail.com wrote: I have just seen the typo below at http://kafka.apache.org/documentation.html . It's supposed to be JMX instead of JMZ, right? []'s Eddie

Re: [DISCUSS] KIP-27 - Conditional Publish

2015-08-10 Thread Flavio Junqueira
I've been trying to understand what is being proposed in this KIP and I've put down some notes with some feedback from Ben that I wanted to share for feedback. I'm not really following the flow of the thread, since I've read a few sources to get to this, and I apologize for that. Here is how I

Re: Review Request 28096: Patch for KAFKA-313

2015-08-10 Thread Ashish Singh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28096/ --- (Updated Aug. 10, 2015, 7:58 p.m.) Review request for kafka, Gwen Shapira,

Re: [DISCUSSION] KIP-29 - Add an IsrPropagateIntervalMs config to KafkaConfig

2015-08-10 Thread Gwen Shapira
An easier (although less clean) solution can be a hidden configuration: Avoid adding the new configuration to KafkaConfig's ConfigDef and the docs, but grab a pre-defined parameter anyway (if exists, use the reasonable default if it doesn't). This will allow us to override the value in tests (and

Re: [jira] [Commented] (KAFKA-1690) new java producer needs ssl support as a client

2015-08-10 Thread Rajini Sivaram
Harsha, I am using the code from https://github.com/harshach/kafka/tree/KAFKA-1690-V1 with the latest commit on 25 July which I think corresponds to the latest patch in KAFKA-1690. Is that correct? I have run SSLConsumerTest, SSLProducerSendTest and SSLSelectorTest several times and the only one

Re: Typo on documentation

2015-08-10 Thread Edward Ribeiro
Okay. On Mon, Aug 10, 2015 at 5:21 PM, Gwen Shapira g...@confluent.io wrote: yeppers. JIRA and patch? On Mon, Aug 10, 2015 at 12:36 PM, Edward Ribeiro edward.ribe...@gmail.com wrote: I have just seen the typo below at http://kafka.apache.org/documentation.html . It's supposed to be

[jira] [Updated] (KAFKA-2418) Typo on official KAFKA documentation

2015-08-10 Thread Edward Ribeiro (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Ribeiro updated KAFKA-2418: -- Affects Version/s: 0.8.0 0.8.1 0.8.2.0 Typo on

[jira] [Updated] (KAFKA-2418) Typo on official KAFKA documentation

2015-08-10 Thread Edward Ribeiro (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Ribeiro updated KAFKA-2418: -- Attachment: KAFKA-2418.patch Adding a patch to web site documentation. Typo on official KAFKA

[jira] [Updated] (KAFKA-2418) Typo on official KAFKA documentation

2015-08-10 Thread Edward Ribeiro (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Ribeiro updated KAFKA-2418: -- Status: Patch Available (was: Open) Typo on official KAFKA documentation

Re: Review Request 33378: Patch for KAFKA-2136

2015-08-10 Thread Aditya Auradkar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33378/#review94822 --- core/src/main/scala/kafka/server/ReplicaManager.scala (line 312)

Re: Review Request 36858: Patch for KAFKA-2120

2015-08-10 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36858/#review94824 ---

Re: Review Request 36858: Patch for KAFKA-2120

2015-08-10 Thread Edward Ribeiro
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36858/#review94817 ---

[jira] [Updated] (KAFKA-2281) org.apache.kafka.clients.producer.internals.ErrorLoggingCallback holds unnecessary byte[] value

2015-08-10 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2281: - Fix Version/s: 0.8.3

Re: Review Request 28096: Patch for KAFKA-313

2015-08-10 Thread Gwen Shapira
On July 29, 2015, 6:35 p.m., Gwen Shapira wrote: core/src/main/scala/kafka/admin/ConsumerGroupCommand.scala, lines 237-242 https://reviews.apache.org/r/28096/diff/4/?file=991387#file991387line237 These look identical - copy/paste error? Ashish Singh wrote: Not really. There

Re: [DISCUSSION] KIP-29 - Add an IsrPropagateIntervalMs config to KafkaConfig

2015-08-10 Thread Jiangjie Qin
Jay, Agreed that we should avoid such configurations if possible. Besides that I think we should also try to avoid super complicated implementation that can potentially cause even more problems. Ashish, For (1), it is basically the group commit Jay recommended. The implementation might be a

[jira] [Updated] (KAFKA-2084) byte rate metrics per client ID (producer and consumer)

2015-08-10 Thread Aditya A Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya A Auradkar updated KAFKA-2084: - Attachment: KAFKA-2084_2015-08-10_13:48:50.patch byte rate metrics per client ID

Re: Review Request 33049: Patch for KAFKA-2084

2015-08-10 Thread Aditya Auradkar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33049/ --- (Updated Aug. 10, 2015, 8:49 p.m.) Review request for kafka, Joel Koshy and

[jira] [Updated] (KAFKA-1929) Convert core kafka module to use the errors in org.apache.kafka.common.errors

2015-08-10 Thread Jeff Holoman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Holoman updated KAFKA-1929: Assignee: Grant Henke (was: Jeff Holoman) Convert core kafka module to use the errors in

Re: Review Request 35437: Patch for KAFKA-2202

2015-08-10 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35437/#review94866 --- Ship it! Ship It! - Guozhang Wang On June 14, 2015, 11:27 a.m.,

[jira] [Updated] (KAFKA-2202) ConsumerPerformance reports a throughput much higher than the actual one

2015-08-10 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2202: - Resolution: Fixed Fix Version/s: 0.8.3 Status: Resolved (was: Patch Available)

[jira] [Resolved] (KAFKA-1828) [ConsumerPerformance] the test result is negative number

2015-08-10 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-1828. -- Resolution: Duplicate [ConsumerPerformance] the test result is negative number

[jira] [Commented] (KAFKA-2202) ConsumerPerformance reports a throughput much higher than the actual one

2015-08-10 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681204#comment-14681204 ] Guozhang Wang commented on KAFKA-2202: -- Thanks for the patch [~omkreddy], +1 and

[jira] [Created] (KAFKA-2419) Allow certain Sensors to be garbage collected after inactivity

2015-08-10 Thread Aditya Auradkar (JIRA)
Aditya Auradkar created KAFKA-2419: -- Summary: Allow certain Sensors to be garbage collected after inactivity Key: KAFKA-2419 URL: https://issues.apache.org/jira/browse/KAFKA-2419 Project: Kafka

Re: Review Request 33049: Patch for KAFKA-2084

2015-08-10 Thread Aditya Auradkar
On Aug. 6, 2015, 4:17 p.m., Jun Rao wrote: A few more comments. We need to be careful with sensors at the client-id level. Clients can come and go (e.g. console consumer). We probably don't want to hold sensors that are not longer actively used since it takes memory. So, we will

[jira] [Updated] (KAFKA-2419) Allow certain Sensors to be garbage collected after inactivity

2015-08-10 Thread Aditya Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-2419: --- Description: Currently, metrics cannot be removed once registered. Implement a feature to

Re: Review Request 33049: Patch for KAFKA-2084

2015-08-10 Thread Aditya Auradkar
On Aug. 6, 2015, 2:02 a.m., Jun Rao wrote: clients/src/main/java/org/apache/kafka/common/metrics/Sensor.java, lines 135-139 https://reviews.apache.org/r/33049/diff/18/?file=1032143#file1032143line135 Is that calculation here right? Based on the calculation in Throttler, it

[jira] [Created] (KAFKA-2420) Merge the Throttle time computation for Quotas and Throttler

2015-08-10 Thread Aditya Auradkar (JIRA)
Aditya Auradkar created KAFKA-2420: -- Summary: Merge the Throttle time computation for Quotas and Throttler Key: KAFKA-2420 URL: https://issues.apache.org/jira/browse/KAFKA-2420 Project: Kafka

[jira] [Commented] (KAFKA-2084) byte rate metrics per client ID (producer and consumer)

2015-08-10 Thread Aditya A Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680741#comment-14680741 ] Aditya A Auradkar commented on KAFKA-2084: -- Updated reviewboard

Re: Review Request 33049: Patch for KAFKA-2084

2015-08-10 Thread Aditya Auradkar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33049/ --- (Updated Aug. 10, 2015, 8:48 p.m.) Review request for kafka, Joel Koshy and

Re: Review Request 33049: Patch for KAFKA-2084

2015-08-10 Thread Jun Rao
On Aug. 6, 2015, 2:02 a.m., Jun Rao wrote: clients/src/main/java/org/apache/kafka/common/metrics/Sensor.java, lines 135-139 https://reviews.apache.org/r/33049/diff/18/?file=1032143#file1032143line135 Is that calculation here right? Based on the calculation in Throttler, it

Re: Review Request 33049: Patch for KAFKA-2084

2015-08-10 Thread Jun Rao
On Aug. 6, 2015, 4:17 p.m., Jun Rao wrote: A few more comments. We need to be careful with sensors at the client-id level. Clients can come and go (e.g. console consumer). We probably don't want to hold sensors that are not longer actively used since it takes memory. So, we will

[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient

2015-08-10 Thread Mayuresh Gharat (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681145#comment-14681145 ] Mayuresh Gharat commented on KAFKA-2120: Updated reviewboard

Re: Review Request 36858: Patch for KAFKA-2120

2015-08-10 Thread Mayuresh Gharat
On Aug. 7, 2015, 12:36 a.m., Jun Rao wrote: clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java, line 223 https://reviews.apache.org/r/36858/diff/3/?file=1024852#file1024852line223 Not sure if the test is needed. First, it seems that batch

Re: Review Request 36858: Patch for KAFKA-2120

2015-08-10 Thread Mayuresh Gharat
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36858/ --- (Updated Aug. 11, 2015, 2:55 a.m.) Review request for kafka. Bugs:

[jira] [Updated] (KAFKA-2120) Add a request timeout to NetworkClient

2015-08-10 Thread Mayuresh Gharat (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayuresh Gharat updated KAFKA-2120: --- Status: Patch Available (was: In Progress) Add a request timeout to NetworkClient

[jira] [Updated] (KAFKA-2120) Add a request timeout to NetworkClient

2015-08-10 Thread Mayuresh Gharat (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayuresh Gharat updated KAFKA-2120: --- Attachment: KAFKA-2120_2015-08-10_19:55:18.patch Add a request timeout to NetworkClient

[jira] [Commented] (KAFKA-1683) Implement a session concept in the socket server

2015-08-10 Thread Eugene Miretsky (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681149#comment-14681149 ] Eugene Miretsky commented on KAFKA-1683: Would this patch include the ability to

Re: Review Request 35437: Patch for KAFKA-2202

2015-08-10 Thread Ewen Cheslack-Postava
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35437/#review94836 --- Ship it! LGTM. Verified it gets rid of KAFKA-1828 too. - Ewen

[jira] [Updated] (KAFKA-2338) Warn users if they change max.message.bytes that they also need to update broker and consumer settings

2015-08-10 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2338: --- Reviewer: Gwen Shapira (was: Jun Rao) Warn users if they change max.message.bytes that they also need to

Re: Review Request 34492: Patch for KAFKA-2210

2015-08-10 Thread Parth Brahmbhatt
On July 28, 2015, 5:18 p.m., Ismael Juma wrote: core/src/main/scala/kafka/common/AuthorizationException.scala, line 24 https://reviews.apache.org/r/34492/diff/9/?file=1018318#file1018318line24 Exceptions without a message are discouraged, so I would remove the no-args

Re: Review Request 34492: Patch for KAFKA-2210

2015-08-10 Thread Edward Ribeiro
On Julho 21, 2015, 1:30 a.m., Edward Ribeiro wrote: core/src/main/scala/kafka/security/auth/Acl.scala, line 71 https://reviews.apache.org/r/34492/diff/8/?file=1017296#file1017296line71 Disclaimer: I am not claiming that you should change the code commented here.

Re: [DISCUSS] KIP-28 - Add a transform client for data processing

2015-08-10 Thread Guozhang Wang
Hello folks, I have updated the KIP page with some detailed API / architecture / packaging proposals, along with the long promised first patch in PR: https://cwiki.apache.org/confluence/display/KAFKA/KIP-28+-+Add+a+processor+client https://github.com/apache/kafka/pull/130 Any feedbacks /

[jira] [Commented] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.

2015-08-10 Thread Parth Brahmbhatt (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681075#comment-14681075 ] Parth Brahmbhatt commented on KAFKA-2210: - Updated reviewboard

[jira] [Updated] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.

2015-08-10 Thread Parth Brahmbhatt (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-2210: Attachment: KAFKA-2210_2015-08-10_18:31:54.patch KafkaAuthorizer: Add all public entities,

Re: Review Request 34492: Patch for KAFKA-2210

2015-08-10 Thread Parth Brahmbhatt
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34492/ --- (Updated Aug. 11, 2015, 1:32 a.m.) Review request for kafka. Bugs:

Re: [jira] [Commented] (KAFKA-1690) new java producer needs ssl support as a client

2015-08-10 Thread Harsha
Thanks for the details. I ran the same test from the branch you used with oracle java 8 , java 7 and IBM java 8 on linux in a loop of 50 didn't see any issue. Can you give me details on which version of IBM jdk did you used and what OS. java -version java version 1.8.0 Java(TM) SE Runtime

[GitHub] kafka pull request: KIP-28: First patch

2015-08-10 Thread guozhangwang
GitHub user guozhangwang opened a pull request: https://github.com/apache/kafka/pull/130 KIP-28: First patch Some open questions collected so far on the first patch. Thanks @gwenshap @jkreps. 1. Can we hide the Chooser interface from users? In other words, if users can

Re: [DISCUSS] KIP-28 - Add a transform client for data processing

2015-08-10 Thread Guozhang Wang
Hi Jiangjie, Not sure I understand the What If user have interleaved groups of messages, each group makes a complete logic? Could you elaborate a bit? About the committing functionality, it currently will only commit up to the processed message's offset; the commit() call it self actually does

Re: [DISCUSS] KIP-28 - Add a transform client for data processing

2015-08-10 Thread Guozhang Wang
Hi Jun, 1. I have removed the streamTime in punctuate() since it is not only triggered by clock time, detailed explanation can be found here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-28+-+Add+a+processor+client#KIP-28-Addaprocessorclient-StreamTime 2. Yes, if users do not schedule

[jira] [Commented] (KAFKA-2411) remove usage of BlockingChannel in the broker

2015-08-10 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680045#comment-14680045 ] ASF GitHub Bot commented on KAFKA-2411: --- GitHub user ijuma opened a pull request:

[GitHub] kafka pull request: KAFKA-2411; [WIP] remove usage of blocking cha...

2015-08-10 Thread ijuma
GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/127 KAFKA-2411; [WIP] remove usage of blocking channel This PR builds on the work from @harshach and only the last commit is relevant. Opening the PR for getting feedback. You can merge this pull request

[jira] [Commented] (KAFKA-2411) remove usage of BlockingChannel in the broker

2015-08-10 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680061#comment-14680061 ] Ismael Juma commented on KAFKA-2411: [~gwenshap], the time has come for you to pass

[jira] [Comment Edited] (KAFKA-2411) remove usage of BlockingChannel in the broker

2015-08-10 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680061#comment-14680061 ] Ismael Juma edited comment on KAFKA-2411 at 8/10/15 1:00 PM: -

[jira] [Updated] (KAFKA-1686) Implement SASL/Kerberos

2015-08-10 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-1686: --- Affects Version/s: (was: 0.9.0) 0.8.2.1 Implement SASL/Kerberos

[jira] [Updated] (KAFKA-1686) Implement SASL/Kerberos

2015-08-10 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-1686: --- Fix Version/s: (was: 0.9.0) 0.8.3 Implement SASL/Kerberos

[jira] [Commented] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.

2015-08-10 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14679758#comment-14679758 ] Ismael Juma commented on KAFKA-2210: Hi [~parth.brahmbhatt], it would be good to get

[jira] [Updated] (KAFKA-2211) KafkaAuthorizer: Add simpleACLAuthorizer implementation.

2015-08-10 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-2211: --- Fix Version/s: 0.8.3 KafkaAuthorizer: Add simpleACLAuthorizer implementation.

[jira] [Commented] (KAFKA-1686) Implement SASL/Kerberos

2015-08-10 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14679764#comment-14679764 ] Ismael Juma commented on KAFKA-1686: [~sriharsha], I changed the target version to

[jira] [Updated] (KAFKA-1683) Implement a session concept in the socket server

2015-08-10 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-1683: --- Fix Version/s: 0.8.3 Implement a session concept in the socket server

Re: [jira] [Commented] (KAFKA-1690) new java producer needs ssl support as a client

2015-08-10 Thread Rajini Sivaram
I was running a Kafka cluster with the latest SSL patch over the weekend with IBM JRE, and it has been running fine without any issues. There was light load on the cluster throughout and intermittent heavy load, all using SSL clients. However I am seeing an intermittent unit test hang in

[jira] [Commented] (KAFKA-2412) Documentation bug: Add information for key.serializer and value.serializer to New Producer Config sections

2015-08-10 Thread darshan kumar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14679886#comment-14679886 ] darshan kumar commented on KAFKA-2412: -- Hi, I have cloned the repository, git clone

[jira] [Commented] (KAFKA-1695) Authenticate connection to Zookeeper

2015-08-10 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680028#comment-14680028 ] Ismael Juma commented on KAFKA-1695: [~parth.brahmbhatt], do you know when the new

[jira] [Updated] (KAFKA-1685) Implement TLS/SSL tests

2015-08-10 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-1685: --- Fix Version/s: (was: 0.9.0) 0.8.3 Implement TLS/SSL tests

[jira] [Updated] (KAFKA-1685) Implement TLS/SSL tests

2015-08-10 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-1685: --- Affects Version/s: (was: 0.9.0) 0.8.2.1 Implement TLS/SSL tests