[jira] [Commented] (KAFKA-2098) Gradle Wrapper Jar gone missing in 0.8.2.1

2015-04-16 Thread Rekha Joshi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14497644#comment-14497644 ] Rekha Joshi commented on KAFKA-2098: Thanks for a prompt reply [~jkreps].What works

[jira] [Commented] (KAFKA-2125) Infinite loop after controlled shutdown succeeds

2015-04-16 Thread Jason Rosenberg (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14497616#comment-14497616 ] Jason Rosenberg commented on KAFKA-2125: So, I'd think the exception there before

[jira] [Commented] (KAFKA-2098) Gradle Wrapper Jar gone missing in 0.8.2.1

2015-04-16 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14497680#comment-14497680 ] Ewen Cheslack-Postava commented on KAFKA-2098: -- The original issue is here:

[jira] [Updated] (KAFKA-2128) kafka.Kafka should return non-zero exit code when caught exception.

2015-04-16 Thread Sasaki Toru (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sasaki Toru updated KAFKA-2128: --- Status: Open (was: Patch Available) kafka.Kafka should return non-zero exit code when caught

[jira] [Updated] (KAFKA-2128) kafka.Kafka should return non-zero exit code when caught exception.

2015-04-16 Thread Sasaki Toru (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sasaki Toru updated KAFKA-2128: --- Status: Patch Available (was: Open) kafka.Kafka should return non-zero exit code when caught

Re: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-16 Thread Guozhang Wang
Hi Adi, 2. I assume you were saying than strictly needed for replications here? Also the concern I have is around error code: today if the replication is not finished within in the replication timeout then the error code will be set accordingly when it returns. Let's say if the produce request

Re: Review Request 33071: Dynamically load JRE-specific class in TestPurgatoryPerformance

2015-04-16 Thread Yasuhiro Matsuda
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33071/#review80342 --- Cool! It looks good to me. - Yasuhiro Matsuda On April 10, 2015,

[jira] [Commented] (KAFKA-2121) error handling in KafkaProducer constructor

2015-04-16 Thread Steven Zhen Wu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498282#comment-14498282 ] Steven Zhen Wu commented on KAFKA-2121: --- Updated reviewboard

[jira] [Updated] (KAFKA-2121) error handling in KafkaProducer constructor

2015-04-16 Thread Steven Zhen Wu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steven Zhen Wu updated KAFKA-2121: -- Attachment: KAFKA-2121_2015-04-16_09:55:14.patch error handling in KafkaProducer constructor

Re: Review Request 33242: Patch for KAFKA-2121

2015-04-16 Thread Steven Wu
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33242/ --- (Updated April 16, 2015, 4:55 p.m.) Review request for kafka. Bugs:

Re: Review Request 33242: Patch for KAFKA-2121

2015-04-16 Thread Steven Wu
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33242/ --- (Updated April 16, 2015, 5:03 p.m.) Review request for kafka. Changes

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations (Thread 2)

2015-04-16 Thread Joe Stein
1. agreed 2. agree new error 3. having discrete operations for tasks makes sense, combining them is confusing for users I think. + 1 for let user change only one thing at a time 4. lets be consistent both to the new code and existing code. lets not confuse the user but give them the right error

[DISCUSSION] KIP-11: ACL Management

2015-04-16 Thread Gwen Shapira
Hi Kafka Authorization Fans, I'm starting a new thread on a specific sub-topic of KIP-11, since this is a bit long :) Currently KIP-11, as I understand it, proposes: * Authorizers are pluggable, with Kafka providing DefaultAuthorizer. * Kafka tools allow adding / managing ACLs. * Those ACLs are

[jira] [Commented] (KAFKA-2098) Gradle Wrapper Jar gone missing in 0.8.2.1

2015-04-16 Thread alexcb (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498307#comment-14498307 ] alexcb commented on KAFKA-2098: --- Thanks for the notes everyone. gradlew (the gradle

[jira] [Updated] (KAFKA-2128) kafka.Kafka should return non-zero exit code when caught exception.

2015-04-16 Thread Sasaki Toru (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sasaki Toru updated KAFKA-2128: --- Attachment: KAFKA-2128-1.patch Upload patch to improve this problem. kafka.Kafka should return

[jira] [Created] (KAFKA-2128) kafka.Kafka should return non-zero exit code when caught exception.

2015-04-16 Thread Sasaki Toru (JIRA)
Sasaki Toru created KAFKA-2128: -- Summary: kafka.Kafka should return non-zero exit code when caught exception. Key: KAFKA-2128 URL: https://issues.apache.org/jira/browse/KAFKA-2128 Project: Kafka

[jira] [Updated] (KAFKA-2128) kafka.Kafka should return non-zero exit code when caught exception.

2015-04-16 Thread Sasaki Toru (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sasaki Toru updated KAFKA-2128: --- Status: Patch Available (was: Open) kafka.Kafka should return non-zero exit code when caught

Re: Review Request 33242: Patch for KAFKA-2121

2015-04-16 Thread Steven Wu
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33242/ --- (Updated April 16, 2015, 5:44 p.m.) Review request for kafka. Bugs:

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-04-16 Thread Jiangjie Qin
Hi Harsha, Took a quick look at the patch. I think it is still a little bit different. KAFKA-1788 only handles the case where a batch sitting in accumulator for too long. The KIP is trying to solve the issue where a batch has already been drained from accumulator and sent to broker. We might be

[jira] [Updated] (KAFKA-2113) TestPurgatoryPerformance does not compile using IBM JDK

2015-04-16 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2113: --- Resolution: Fixed Fix Version/s: 0.8.3 Status: Resolved (was: Patch Available) Thanks for

[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient

2015-04-16 Thread Suresh Srinivas (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498340#comment-14498340 ] Suresh Srinivas commented on KAFKA-2120: KAFKA-1788 which has been in the work for

Re: Review Request 33242: Patch for KAFKA-2121

2015-04-16 Thread Ewen Cheslack-Postava
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33242/#review80346 --- Looks good, left a few comments. KafkaConsumer suffers from this

[jira] [Updated] (KAFKA-2121) error handling in KafkaProducer constructor

2015-04-16 Thread Steven Zhen Wu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steven Zhen Wu updated KAFKA-2121: -- Attachment: KAFKA-2121_2015-04-16_10:43:55.patch error handling in KafkaProducer constructor

Re: Review Request 33242: Patch for KAFKA-2121

2015-04-16 Thread Steven Wu
On April 16, 2015, 5:29 p.m., Ewen Cheslack-Postava wrote: clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java, line 531 https://reviews.apache.org/r/33242/diff/2/?file=931792#file931792line531 This code is all single threaded, is the AtomicReference really

[jira] [Commented] (KAFKA-2121) error handling in KafkaProducer constructor

2015-04-16 Thread Steven Zhen Wu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498371#comment-14498371 ] Steven Zhen Wu commented on KAFKA-2121: --- Updated reviewboard

Re: [DISCUSS] New consumer offset commit API

2015-04-16 Thread Jiangjie Qin
Hey Ewen, This makes sense. People usually do not want to stop consuming when committing offsets. One corner case about async commit with retries I am thinking is that it is possible that two offset commits interleave with each other and that might create problem. Like you said maybe we can

[jira] [Updated] (KAFKA-2121) error handling in KafkaProducer constructor

2015-04-16 Thread Steven Zhen Wu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steven Zhen Wu updated KAFKA-2121: -- Attachment: (was: KAFKA-2121.patch.2) error handling in KafkaProducer constructor

[jira] [Commented] (KAFKA-2098) Gradle Wrapper Jar gone missing in 0.8.2.1

2015-04-16 Thread alexcb (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498750#comment-14498750 ] alexcb commented on KAFKA-2098: --- Perhaps I was unclear in what I was suggesting. Since

[jira] [Commented] (KAFKA-2098) Gradle Wrapper Jar gone missing in 0.8.2.1

2015-04-16 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498774#comment-14498774 ] Ewen Cheslack-Postava commented on KAFKA-2098: -- Sorry @alexcb, I think we

RE: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-16 Thread Aditya Auradkar
Hey Guozhang, I don't think we should return an error if the request is satisfied after Y (throttling timeout) because it may cause the producer to think that the request was not ack'ed at all. Aditya From: Guozhang Wang [wangg...@gmail.com] Sent:

Re: [KIP-DISCUSSION] KIP-13 Quotas

2015-04-16 Thread Jun Rao
The quota check for the fetch request is a bit different from the produce request. I assume that for the fetch request, we will first get an estimated fetch response size to do the quota check. There are two things to think about. First, when we actually send the response, we probably don't want

Re: Review Request 33242: Patch for KAFKA-2121

2015-04-16 Thread Steven Wu
On April 16, 2015, 5:29 p.m., Ewen Cheslack-Postava wrote: Looks good, left a few comments. KafkaConsumer suffers from this same problem. Patching that should be pretty much identical -- any chance you could extend this to cover that as well? sure. I can extend this to

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-04-16 Thread Jiangjie Qin
Hi Guozhang, By implicit timeout for close() and flush(), I meant that currently we don’t have a explicit timeout for close() or flush() when a broker is down, so they can take pretty long up to TCP timeout which is hours as you mentioned. With the client side request timeout, the waiting time

Re: [DISCUSS] KIP-4 - Command line and centralized administrative operations (Thread 2)

2015-04-16 Thread Jun Rao
1. For the lags, we can add a new field lags per partition. It will return for each replica that's not in isr, the replica id and the lag in messages. Also, if TMR is sent to a non-leader, the response can just include an empty array for isr and lags. 2. So, we will just return a topic level

Re: Review Request 33239: Patch for KAFKA-2126

2015-04-16 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33239/#review80394 --- Ship it! Ship It! - Jiangjie Qin On April 15, 2015, 8:19 p.m.,

Re: [DISCUSSION] KIP-11: ACL Management

2015-04-16 Thread Jun Rao
Hi, Gwen, What you suggested seems reasonable. I guess we will need the Principal, Privilege pair and the Resource in grant() and revoke()? Is the Hive authorization api the following? It's weird that it takes user in checkPermissions(), but not in authorize().

[jira] [Created] (KAFKA-2129) Consumer could make multiple concurrent metadata requests

2015-04-16 Thread Tim Brooks (JIRA)
Tim Brooks created KAFKA-2129: - Summary: Consumer could make multiple concurrent metadata requests Key: KAFKA-2129 URL: https://issues.apache.org/jira/browse/KAFKA-2129 Project: Kafka Issue

Re: Review Request 33242: Patch for KAFKA-2121

2015-04-16 Thread Ewen Cheslack-Postava
On April 16, 2015, 5:29 p.m., Ewen Cheslack-Postava wrote: clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java, line 548 https://reviews.apache.org/r/33242/diff/2/?file=931792#file931792line548 One idea for making this less verbose and redundant: make all

[jira] [Commented] (KAFKA-2098) Gradle Wrapper Jar gone missing in 0.8.2.1

2015-04-16 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498431#comment-14498431 ] Ewen Cheslack-Postava commented on KAFKA-2098: -- [~alexcb] That works as long

Build failed in Jenkins: Kafka-trunk #464

2015-04-16 Thread Apache Jenkins Server
See https://builds.apache.org/job/Kafka-trunk/464/changes Changes: [junrao] kafka-2113; TestPurgatoryPerformance does not compile using IBM JDK; patched by Rajini Sivaram; reviewed by Yasuhiro Matsuda and Jun Rao -- [...truncated 1245 lines...]

Re: [DISCUSSION] KIP-11: ACL Management

2015-04-16 Thread Gari Singh
Hi Gwen - I tend to agree with your proposal. As you mention the exact details / interfaces would need to be worked out, but this would be more in line with how JAAS and JACC work in the Java / JEE worlds. I do think that it might be nice to include / provide some helper APIs / methods for

[jira] [Updated] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.

2015-04-16 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-1660: Attachment: KAFKA-1660_2015-04-16_11:35:37.patch Ability to call close() with a timeout on the

[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.

2015-04-16 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498469#comment-14498469 ] Jiangjie Qin commented on KAFKA-1660: - Updated reviewboard

Re: Review Request 31850: Patch for KAFKA-1660

2015-04-16 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31850/ --- (Updated April 16, 2015, 6:35 p.m.) Review request for kafka. Bugs:

Re: Review Request 31850: Patch for KAFKA-1660

2015-04-16 Thread Jiangjie Qin
On April 11, 2015, 8:02 p.m., Jay Kreps wrote: clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java, line 155 https://reviews.apache.org/r/31850/diff/8/?file=923927#file923927line155 This scheme is clever but non-obvious, is there a simpler