Re: Review Request 33065: Patch for KAFKA-1928

2015-05-13 Thread Gwen Shapira
On May 6, 2015, 10:50 p.m., Jun Rao wrote: clients/src/main/java/org/apache/kafka/common/network/Selector.java, lines 198-199 https://reviews.apache.org/r/33065/diff/4/?file=947532#file947532line198 Could we just iterate the keySet directly instead of making a copy first? We

Writing a client: Connection pooling

2015-05-13 Thread Warren Falk
I'm working on the C# client. The current Kafka Protocol page says this: it should not generally be necessary to maintain multiple connections to a single broker from a single client instance (i.e. connection pooling) But then says this: The server guarantees that on a single TCP connection,

[jira] [Commented] (KAFKA-1682) Security for Kafka

2015-05-13 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542422#comment-14542422 ] Jun Rao commented on KAFKA-1682: 3. The SSL work is now being done in KAFKA-1690, on top

Re: KIP Hangout notes

2015-05-13 Thread Jun Rao
Michael, The ssl work in now being done in https://issues.apache.org/jira/browse/KAFKA-1690. It would be great if you can help review it too. Also updated the top level security jira accordingly. Thanks, Jun On Wed, May 13, 2015 at 11:23 AM, Michael Herstine mherst...@linkedin.com.invalid

Re: Message corruption with new Java client + snappy + broker restart

2015-05-13 Thread Jun Rao
Roger, For 1), the producer actually doesn't move buffered data around during leadership change. The producer buffer is per partition, independent of what the current leader of that partition is. Does the issue happen w/o retry? Does this happen in a particular version of snappy? For 2),

Re: Review Request 33620: Patch for KAFKA-1690

2015-05-13 Thread Jun Rao
On May 13, 2015, 3:40 p.m., Jun Rao wrote: clients/src/main/java/org/apache/kafka/common/network/Selector.java, lines 286-291 https://reviews.apache.org/r/33620/diff/5/?file=957067#file957067line286 I am a bit confused how this works during the handshake. During the handshake,

[jira] [Commented] (KAFKA-1690) new java producer needs ssl support as a client

2015-05-13 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542437#comment-14542437 ] Sriharsha Chintalapani commented on KAFKA-1690: --- [~junrao] Supporting

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations (Thread 2)

2015-05-13 Thread Joel Koshy
Just had a few minor questions before I join the vote thread. Apologies if these have been discussed: - Do we need DecreasePartitionsNotAllowed? i.e., can we just return InvalidPartitions instead? - AdminClient.listTopics: should we allow listing all partitions? Or do you intend for the

[jira] [Commented] (KAFKA-790) Kafka server throws UnknownTopicOrPartitionException right after becoming leader for a new partition

2015-05-13 Thread saiprasad mishra (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542808#comment-14542808 ] saiprasad mishra commented on KAFKA-790: [~nehanarkhede] I know its kind of later

Re: Review Request 34170: Patch for KAFKA-2191

2015-05-13 Thread Dong Lin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34170/ --- (Updated May 13, 2015, 10:32 p.m.) Review request for kafka. Bugs:

Re: Review Request 34170: Patch for KAFKA-2191

2015-05-13 Thread Dong Lin
On May 13, 2015, 5:14 p.m., Jay Kreps wrote: clients/src/main/java/org/apache/kafka/common/metrics/stats/SampledStat.java, line 126 https://reviews.apache.org/r/34170/diff/1/?file=958216#file958216line126 how come we are removing the event count threshold. Oh.. My bad, I should

Re: Review Request 34170: Patch for KAFKA-2191

2015-05-13 Thread Aditya Auradkar
On May 13, 2015, 5:14 p.m., Jay Kreps wrote: clients/src/main/java/org/apache/kafka/common/metrics/stats/Rate.java, line 62 https://reviews.apache.org/r/34170/diff/1/?file=958215#file958215line62 Is this actually right? I agree you'll get discontinuities as the measured time

[jira] [Updated] (KAFKA-2191) Measured rate should not be infinite

2015-05-13 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-2191: Attachment: KAFKA-2191_2015-05-13_15:32:15.patch Measured rate should not be infinite

[jira] [Commented] (KAFKA-2191) Measured rate should not be infinite

2015-05-13 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542851#comment-14542851 ] Dong Lin commented on KAFKA-2191: - Updated reviewboard

Re: Review Request 34170: Patch for KAFKA-2191

2015-05-13 Thread Jay Kreps
On May 13, 2015, 11:50 p.m., Aditya Auradkar wrote: clients/src/main/java/org/apache/kafka/common/metrics/stats/SampledStat.java, line 45 https://reviews.apache.org/r/34170/diff/2/?file=958605#file958605line45 I think this is a good catch. Just so I understand, this

Re: Review Request 34170: Patch for KAFKA-2191

2015-05-13 Thread Jay Kreps
On May 13, 2015, 11:50 p.m., Aditya Auradkar wrote: clients/src/main/java/org/apache/kafka/common/metrics/stats/SampledStat.java, line 45 https://reviews.apache.org/r/34170/diff/2/?file=958605#file958605line45 I think this is a good catch. Just so I understand, this

Re: [DISCUSS] Add missing API to old high level consumer

2015-05-13 Thread Joe Stein
My gut reaction is that this isn't that important for folks otherwise they would have complained already. If it is a blocker for folks upgrading to 0.8.2.1 then we should do a 0.8.2.2 release with this fix in it. For 0.9.0. we are pushing for folks to start using the new consumer and that is the

Re: Review Request 33620: Patch for KAFKA-1690

2015-05-13 Thread Sriharsha Chintalapani
On May 13, 2015, 3:40 p.m., Jun Rao wrote: clients/src/main/java/org/apache/kafka/common/network/TransportLayer.java, line 98 https://reviews.apache.org/r/33620/diff/5/?file=957068#file957068line98 This seems to be ssl specific. Is this needed in the interface? It doesn't seem

Re: Review Request 34170: Patch for KAFKA-2191

2015-05-13 Thread Aditya Auradkar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34170/#review83687 ---

Re: Writing a client: Connection pooling

2015-05-13 Thread Guozhang Wang
Hello Warren, I seems your C# client is both a producer and a consumer. Then with the behavior of the broker, your suspension is correct that a long pooling fetch using the same TCP connection will block subsequent produce / metadata requests. I think the statement that it should not generally

Re: Review Request 33620: Patch for KAFKA-1690

2015-05-13 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33620/#review83322 --- Thanks for the patch. A few comments below. The following unit

[jira] [Commented] (KAFKA-2191) Measured rate should not be infinite

2015-05-13 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542135#comment-14542135 ] Dong Lin commented on KAFKA-2191: - Created reviewboard

[jira] [Updated] (KAFKA-2191) Measured rate should not be infinite

2015-05-13 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-2191: Status: Patch Available (was: Open) Measured rate should not be infinite

[jira] [Updated] (KAFKA-2191) Measured rate should not be infinite

2015-05-13 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-2191: Attachment: KAFKA-2191.patch Measured rate should not be infinite

Review Request 34170: Patch for KAFKA-2191

2015-05-13 Thread Dong Lin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34170/ --- Review request for kafka. Bugs: KAFKA-2191

[jira] [Commented] (KAFKA-1690) new java producer needs ssl support as a client

2015-05-13 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542145#comment-14542145 ] Jun Rao commented on KAFKA-1690: Also, could anyone comment on whether we need to support

Re: Message corruption with new Java client + snappy + broker restart

2015-05-13 Thread Guozhang Wang
Not sure if it is related, but we could verify it with Roger's scenario. Roger, what is your snappy version? Could you try version 1.1.1.7 and see if this issue still persists? Guozhang On Tue, May 12, 2015 at 10:01 PM, Jiangjie Qin j...@linkedin.com.invalid wrote: Does this related to

Re: Review Request 34170: Patch for KAFKA-2191

2015-05-13 Thread Jay Kreps
On May 13, 2015, 5:14 p.m., Jay Kreps wrote: clients/src/main/java/org/apache/kafka/common/metrics/stats/Rate.java, line 62 https://reviews.apache.org/r/34170/diff/1/?file=958215#file958215line62 Is this actually right? I agree you'll get discontinuities as the measured time

Re: Review Request 33731: Second Attempt to Fix KAFKA-2160

2015-05-13 Thread Jay Kreps
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33731/#review83707 --- Pool is meant to be a simple, general purpose and very thin wrapper

Re: Review Request 33731: Second Attempt to Fix KAFKA-2160

2015-05-13 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33731/#review83694 --- Thanks for the patch. A few comments. 1. The changes in Pool are

Re: Review Request 34170: Patch for KAFKA-2191

2015-05-13 Thread Jay Kreps
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34170/#review83629 ---

Re: Review Request 33620: Patch for KAFKA-1690

2015-05-13 Thread Sriharsha Chintalapani
On May 13, 2015, 3:40 p.m., Jun Rao wrote: clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java, lines 61-112 https://reviews.apache.org/r/33620/diff/4/?file=955770#file955770line61 Those properties will be used by broker config as well, right? Should we put

Re: Review Request 33620: Patch for KAFKA-1690

2015-05-13 Thread Sriharsha Chintalapani
On May 13, 2015, 3:40 p.m., Jun Rao wrote: Thanks for the patch. A few comments below. The following unit tests seem to fail. org.apache.kafka.common.network.SSLFactoryTest testSSLFactoryConfiguration FAILED java.lang.IllegalArgumentException: TLSv1.2 at

[jira] [Commented] (KAFKA-1690) new java producer needs ssl support as a client

2015-05-13 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542282#comment-14542282 ] Sriharsha Chintalapani commented on KAFKA-1690: --- [~junrao] I think we don't

[jira] [Updated] (KAFKA-1690) new java producer needs ssl support as a client

2015-05-13 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1690: --- Status: In Progress (was: Patch Available) [~rsivaram], could you help review the patch too and especially

[jira] [Updated] (KAFKA-2167) ZkUtils updateEphemeralPath JavaDoc (spelling and correctness)

2015-05-13 Thread Neelesh Srinivas Salian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neelesh Srinivas Salian updated KAFKA-2167: --- Fix Version/s: 0.8.2.0 Status: Patch Available (was: Open) Here

Re: KIP Hangout notes

2015-05-13 Thread Michael Herstine
Regarding the SSL code‹ is there an RB available? I don¹t see a recent patch uploaded to https://issues.apache.org/jira/browse/KAFKA-1684Š How can other folks see the code? On 5/12/15, 12:07 PM, Gwen Shapira gshap...@cloudera.com wrote: My notes from the hangout: * KIP-11: Based on feedback

[jira] [Commented] (KAFKA-2167) ZkUtils updateEphemeralPath JavaDoc (spelling and correctness)

2015-05-13 Thread Neelesh Srinivas Salian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542308#comment-14542308 ] Neelesh Srinivas Salian commented on KAFKA-2167: Updated fix version from

[jira] [Updated] (KAFKA-2167) ZkUtils updateEphemeralPath JavaDoc (spelling and correctness)

2015-05-13 Thread Neelesh Srinivas Salian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neelesh Srinivas Salian updated KAFKA-2167: --- Fix Version/s: (was: 0.8.2.0) 0.8.3 ZkUtils

[GitHub] kafka pull request: remove unnecessary imports

2015-05-13 Thread xuwei-k
GitHub user xuwei-k opened a pull request: https://github.com/apache/kafka/pull/65 remove unnecessary imports You can merge this pull request into a Git repository by running: $ git pull https://github.com/xuwei-k/kafka remove-imports Alternatively you can review and apply

[jira] [Created] (KAFKA-2191) Measured rate should not be infinite

2015-05-13 Thread Dong Lin (JIRA)
Dong Lin created KAFKA-2191: --- Summary: Measured rate should not be infinite Key: KAFKA-2191 URL: https://issues.apache.org/jira/browse/KAFKA-2191 Project: Kafka Issue Type: Bug

[jira] [Updated] (KAFKA-2176) DefaultPartitioner doesn't perform consistent hashing based on

2015-05-13 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Maravić updated KAFKA-2176: Status: Open (was: Patch Available) DefaultPartitioner doesn't perform consistent hashing based

[jira] [Commented] (KAFKA-2176) DefaultPartitioner doesn't perform consistent hashing based on

2015-05-13 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541484#comment-14541484 ] Igor Maravić commented on KAFKA-2176: - Partitioning is working as expected with new

[jira] [Resolved] (KAFKA-2176) DefaultPartitioner doesn't perform consistent hashing based on

2015-05-13 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Maravić resolved KAFKA-2176. - Resolution: Won't Fix DefaultPartitioner doesn't perform consistent hashing based on

[jira] [Commented] (KAFKA-1911) Log deletion on stopping replicas should be async

2015-05-13 Thread antoine vianey (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541498#comment-14541498 ] antoine vianey commented on KAFKA-1911: --- it should handle the case of a producer

[jira] [Commented] (KAFKA-2169) Upgrade to zkclient-0.5

2015-05-13 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542708#comment-14542708 ] Jun Rao commented on KAFKA-2169: Thought a bit more about this. When handling