Re: Review Request 34241: Patch for KAFKA-2190

2015-05-19 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34241/#review84276 --- Ship it! core/src/main/scala/kafka/tools/MirrorMaker.scala

Re: Review Request 24214: Patch for KAFKA-1374

2015-05-19 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24214/#review84278 --- Thanks for the updated patch. This looks good. I ended up rebasing

Review Request 34397: Patch for KAFKA-1374

2015-05-19 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34397/ --- Review request for kafka. Bugs: KAFKA-1374

[jira] [Updated] (KAFKA-1374) LogCleaner (compaction) does not support compressed topics

2015-05-19 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1374: -- Attachment: KAFKA-1374.patch LogCleaner (compaction) does not support compressed topics

[jira] [Commented] (KAFKA-1374) LogCleaner (compaction) does not support compressed topics

2015-05-19 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14549998#comment-14549998 ] Joel Koshy commented on KAFKA-1374: --- Created reviewboard

Re: Review Request 33065: Patch for KAFKA-1928

2015-05-19 Thread Gwen Shapira
On May 18, 2015, 4:53 p.m., Jun Rao wrote: clients/src/main/java/org/apache/kafka/common/network/Selector.java, line 115 https://reviews.apache.org/r/33065/diff/12/?file=962798#file962798line115 Do we need to turn this into a map? NetworkReceive already has the connection id.

[jira] [Commented] (KAFKA-1928) Move kafka.network over to using the network classes in org.apache.kafka.common.network

2015-05-19 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550030#comment-14550030 ] Gwen Shapira commented on KAFKA-1928: - Updated reviewboard

[jira] [Updated] (KAFKA-1928) Move kafka.network over to using the network classes in org.apache.kafka.common.network

2015-05-19 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1928: Attachment: KAFKA-1928_2015-05-19_11:26:18.patch Move kafka.network over to using the network

[jira] [Commented] (KAFKA-1778) Create new re-elect controller admin function

2015-05-19 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550019#comment-14550019 ] Joel Koshy commented on KAFKA-1778: --- Jun - I think this is more for

[jira] [Commented] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550120#comment-14550120 ] Manikumar Reddy commented on KAFKA-2198: Case 1: Zookeeper chroot path is

Re: Review Request 33065: Patch for KAFKA-1928

2015-05-19 Thread Gwen Shapira
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33065/ --- (Updated May 19, 2015, 8:26 a.m.) Review request for kafka. Bugs: 1928 and

[jira] [Commented] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550100#comment-14550100 ] Manikumar Reddy commented on KAFKA-2198: Created reviewboard

Review Request 34403: Patch for KAFKA-2198

2015-05-19 Thread Manikumar Reddy O
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34403/ --- Review request for kafka. Bugs: KAFKA-2198

[jira] [Updated] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-2198: --- Assignee: Manikumar Reddy Status: Patch Available (was: Open) kafka-topics.sh exits

[jira] [Updated] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-2198: --- Attachment: KAFKA-2198.patch kafka-topics.sh exits with 0 status on failures

[jira] [Created] (KAFKA-2201) Open file handle leak

2015-05-19 Thread Albert Visagie (JIRA)
Albert Visagie created KAFKA-2201: - Summary: Open file handle leak Key: KAFKA-2201 URL: https://issues.apache.org/jira/browse/KAFKA-2201 Project: Kafka Issue Type: Bug Components:

[jira] [Commented] (KAFKA-2134) Producer blocked on metric publish

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550190#comment-14550190 ] Manikumar Reddy commented on KAFKA-2134: metricChange() method will be called

Build failed in Jenkins: KafkaPreCommit #104

2015-05-19 Thread Apache Jenkins Server
See https://builds.apache.org/job/KafkaPreCommit/104/changes Changes: [wangguoz] KAFKA-2196; Remove identical topic constraint in round-robin assignor; reviewed by Guozhang Wang -- [...truncated 769 lines...] at

[jira] [Updated] (KAFKA-2196) remove roundrobin identical topic constraint in consumer coordinator

2015-05-19 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2196: - Resolution: Fixed Reviewer: Guozhang Wang (was: Joel Koshy) Status: Resolved (was:

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Ewen Cheslack-Postava
The fact that I understand the producer internals and am still struggling to understand the implications of the different settings, how I would set them, and how they potentially interact such that I could set invalid combinations seems like a red flag to me... Being able to say I want produce

Re: Review Request 33620: Patch for KAFKA-1690

2015-05-19 Thread Sriharsha Chintalapani
On May 15, 2015, 8:26 p.m., Michael Herstine wrote: clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java, line 418 https://reviews.apache.org/r/33620/diff/5/?file=957065#file957065line418 So you're transferring bytes from `appReadBuffer` to `dst`, right?

[jira] [Updated] (KAFKA-2191) Measured rate should not be infinite

2015-05-19 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2191: - Reviewer: Joel Koshy (was: Guozhang Wang) [~jjkoshy] Could you help taking a look at this one or

RE: [VOTE] KIP-21 Dynamic Configuration

2015-05-19 Thread Aditya Auradkar
Thanks Andrii. I'll make the changes. I've also updated KIP-21 to include the new config requests. Take a look and vote. https://cwiki.apache.org/confluence/display/KAFKA/KIP-21+-+Dynamic+Configuration Aditya From: Andrii Biletskyi

[jira] [Created] (KAFKA-2206) Add AlterConfig and DescribeConfig requests to Kafka

2015-05-19 Thread Aditya Auradkar (JIRA)
Aditya Auradkar created KAFKA-2206: -- Summary: Add AlterConfig and DescribeConfig requests to Kafka Key: KAFKA-2206 URL: https://issues.apache.org/jira/browse/KAFKA-2206 Project: Kafka Issue

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Jiangjie Qin
Hey Mayuresh, I think our purpose is to find out the simplest user interface but still support all the flexibilities user might need. In that sense, we have to expose 1. linger.ms for batching purpose, 2. request timeout to support different RTT. 3. Some blocking timeout for send() Etc,.. So

Re: Review Request 33065: Patch for KAFKA-1928

2015-05-19 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33065/#review84398 --- Thanks for the latest patch. Only a few more minor comments. Also,

Re: Review Request 33065: Patch for KAFKA-1928

2015-05-19 Thread Jun Rao
On May 18, 2015, 4:53 p.m., Jun Rao wrote: clients/src/main/java/org/apache/kafka/common/network/Send.java, line 56 https://reviews.apache.org/r/33065/diff/12/?file=962799#file962799line56 This needs to be long since NetworkSend always includes a size buffer, plus the payload

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Jiangjie Qin
Hi Jay, I updated what I think int KIP wiki. Just a short summary here. Because we need timeout for: 1. Send() 2. Batches in accumulator 3. Requests in flight. That means we need to have at least three configurations if we do not reuse configurations. I think we probably want to also separate

Re: Review Request 34397: Patch for KAFKA-1374

2015-05-19 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34397/ --- (Updated May 20, 2015, 12:20 a.m.) Review request for kafka. Bugs:

[jira] [Commented] (KAFKA-1374) LogCleaner (compaction) does not support compressed topics

2015-05-19 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551535#comment-14551535 ] Joel Koshy commented on KAFKA-1374: --- Updated reviewboard

Re: Review Request 34397: Patch for KAFKA-1374

2015-05-19 Thread Joel Koshy
On May 19, 2015, 9:41 p.m., Guozhang Wang wrote: About duplicating the compression logic, one way would be: 1. Add the following functions in ByteBufferMessageSet as: fill(buffer: ByteBuffer, compressionCodec: CompressionCodec, messages: MessageAndOffset*): Int 2. Change

[jira] [Updated] (KAFKA-1374) LogCleaner (compaction) does not support compressed topics

2015-05-19 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1374: -- Attachment: KAFKA-1374_2015-05-19_17:20:44.patch LogCleaner (compaction) does not support compressed

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Mayuresh Gharat
Hi Jay, If I am understanding this correctly, we should treat the Batch Timeout separately from the Request Timeout. Request time out can be used only for inflight request (request that have been sent to kafka brokers and waiting for response) and Batch Timeout can be used to expire the batches

[jira] [Commented] (KAFKA-2160) DelayedOperationPurgatory should remove the pair in watchersForKey with empty watchers list

2015-05-19 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551671#comment-14551671 ] Guozhang Wang commented on KAFKA-2160: -- Thanks for the review, committed to trunk.

[jira] [Updated] (KAFKA-2160) DelayedOperationPurgatory should remove the pair in watchersForKey with empty watchers list

2015-05-19 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2160: - Resolution: Fixed Status: Resolved (was: Patch Available) DelayedOperationPurgatory

[jira] [Commented] (KAFKA-2203) Get gradle build to work with Java 8

2015-05-19 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551746#comment-14551746 ] Ewen Cheslack-Postava commented on KAFKA-2203: -- +1 Double checked against

[jira] [Created] (KAFKA-2205) Generalize TopicConfigManager to handle multiple entity configs

2015-05-19 Thread Aditya Auradkar (JIRA)
Aditya Auradkar created KAFKA-2205: -- Summary: Generalize TopicConfigManager to handle multiple entity configs Key: KAFKA-2205 URL: https://issues.apache.org/jira/browse/KAFKA-2205 Project: Kafka

[jira] [Commented] (KAFKA-2017) Persist Coordinator State for Coordinator Failover

2015-05-19 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551540#comment-14551540 ] Guozhang Wang commented on KAFKA-2017: -- Created reviewboard

[jira] [Updated] (KAFKA-2017) Persist Coordinator State for Coordinator Failover

2015-05-19 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2017: - Reviewer: Jun Rao Persist Coordinator State for Coordinator Failover

[jira] [Updated] (KAFKA-2017) Persist Coordinator State for Coordinator Failover

2015-05-19 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2017: - Attachment: KAFKA-2017.patch Persist Coordinator State for Coordinator Failover

[jira] [Updated] (KAFKA-2017) Persist Coordinator State for Coordinator Failover

2015-05-19 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2017: - Assignee: Guozhang Wang (was: Onur Karaman) Status: Patch Available (was: Open) Persist

Re: Review Request 33620: Patch for KAFKA-1690

2015-05-19 Thread Sriharsha Chintalapani
On May 15, 2015, 8:26 p.m., Michael Herstine wrote: clients/src/main/java/org/apache/kafka/common/network/SSLTransportLayer.java, lines 112-121 https://reviews.apache.org/r/33620/diff/5/?file=957065#file957065line112 Sorry, but I'm a little confused here. Why would the caller be

[jira] [Commented] (KAFKA-2147) Unbalanced replication can cause extreme purgatory growth

2015-05-19 Thread Craig Allsop (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551556#comment-14551556 ] Craig Allsop commented on KAFKA-2147: - [~junrao] Forgive me if this is the wrong

Build failed in Jenkins: KafkaPreCommit #103

2015-05-19 Thread Apache Jenkins Server
See https://builds.apache.org/job/KafkaPreCommit/103/changes Changes: [wangguoz] KAFKA-2160; remove watcher list if empty in purgatory; remove join-group purgatory in coordinator; reviewed by Jun Rao -- [...truncated 1354 lines...]

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Jay Kreps
Yeah I think linger.ms remains separate, setting that is a performance optimization rather than failure handling thing. We should ideally sanity check this, though, in my proposal, since if they set linger.ms request.timeout then that won't work. It's true that in my proposal that the actual

[jira] [Commented] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550411#comment-14550411 ] Manikumar Reddy commented on KAFKA-2198: Agreed. This has been fixed on trunk

[jira] [Commented] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550413#comment-14550413 ] Manikumar Reddy commented on KAFKA-2198: Agreed. This has been fixed on trunk

[jira] [Issue Comment Deleted] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-2198: --- Comment: was deleted (was: Agreed. This has been fixed on trunk (KAFKA-1668). )

[jira] [Issue Comment Deleted] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-2198: --- Comment: was deleted (was: Agreed. This has been fixed on trunk (KAFKA-1668). )

Re: Review Request 34403: Patch for KAFKA-2198

2015-05-19 Thread Manikumar Reddy O
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34403/ --- (Updated May 19, 2015, 12:59 p.m.) Review request for kafka. Bugs:

[jira] [Commented] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550412#comment-14550412 ] Manikumar Reddy commented on KAFKA-2198: Agreed. This has been fixed on trunk

[jira] [Commented] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550407#comment-14550407 ] Manikumar Reddy commented on KAFKA-2198: Updated reviewboard

[jira] [Updated] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-2198: --- Attachment: KAFKA-2198_2015-05-19_18:41:25.patch kafka-topics.sh exits with 0 status on

[jira] [Commented] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Bob Halley (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550352#comment-14550352 ] Bob Halley commented on KAFKA-2198: --- I realize that the chroot path is optional, but

[jira] [Commented] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550387#comment-14550387 ] Manikumar Reddy commented on KAFKA-2198: Updated reviewboard

[jira] [Updated] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-2198: --- Attachment: KAFKA-2198_2015-05-19_18:27:01.patch kafka-topics.sh exits with 0 status on

Re: Review Request 34403: Patch for KAFKA-2198

2015-05-19 Thread Manikumar Reddy O
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34403/ --- (Updated May 19, 2015, 1:14 p.m.) Review request for kafka. Bugs: KAFKA-2198

[jira] [Created] (KAFKA-2202) ConsumerPerformance reports a throughput much higher than the actual one

2015-05-19 Thread JIRA
Micael Capitão created KAFKA-2202: - Summary: ConsumerPerformance reports a throughput much higher than the actual one Key: KAFKA-2202 URL: https://issues.apache.org/jira/browse/KAFKA-2202 Project:

[jira] [Updated] (KAFKA-2191) Measured rate should not be infinite

2015-05-19 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-2191: - Attachment: KAFKA-2191.patch Measured rate should not be infinite

Review Request 34418: Patch for KAFKA-2191

2015-05-19 Thread Jay Kreps
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34418/ --- Review request for kafka. Bugs: KAFKA-2191

Re: Review Request 34273: remove roundrobin identical topic constraint in consumer coordinator

2015-05-19 Thread Onur Karaman
On May 18, 2015, 9:46 p.m., Guozhang Wang wrote: core/src/main/scala/kafka/coordinator/PartitionAssignor.scala, lines 87-89 https://reviews.apache.org/r/34273/diff/1/?file=961395#file961395line87 I feel this logic's irregular assignment result may make it a bit harder to reason

[jira] [Commented] (KAFKA-1778) Create new re-elect controller admin function

2015-05-19 Thread Abhishek Nigam (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550781#comment-14550781 ] Abhishek Nigam commented on KAFKA-1778: --- Jun, The way I see it pinning the

[jira] [Commented] (KAFKA-2191) Measured rate should not be infinite

2015-05-19 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550791#comment-14550791 ] Jay Kreps commented on KAFKA-2191: -- Hey [~lindong] here is a patch that shows what I'm

[jira] [Commented] (KAFKA-2191) Measured rate should not be infinite

2015-05-19 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550787#comment-14550787 ] Jay Kreps commented on KAFKA-2191: -- Created reviewboard

Re: Review Request 34170: Patch for KAFKA-2191

2015-05-19 Thread Jay Kreps
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34170/#review84341 ---

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Mayuresh Gharat
So what I understand is that, we would have 3 time outs : 1) replication timeout 2) request timeout 3) metadata timeout (existing) The request timeout has to be greater than the replication timeout. request timeout is for messages already sent to kafka and the producer is waiting for them.

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Jay Kreps
Here is the concern I had with reusing the metadata.fetch.timeout.ms: Previously people were using this as a bound on the time send() would block. It isn't a bound on the time we will wait on a metadata request, just the time the send call will block if metadata is missing for the topic. We told

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Jay Kreps
So the alternative to consider would be to instead have max.block.ms (or something) request.timeout replication.timeout I think this better captures what the user cares about. Here is how it would work. *max.send.block.ms http://max.send.block.ms* is the bound on the maximum time the

[jira] [Commented] (KAFKA-2195) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest

2015-05-19 Thread Andrii Biletskyi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550682#comment-14550682 ] Andrii Biletskyi commented on KAFKA-2195: - Created reviewboard

[jira] [Updated] (KAFKA-2195) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest

2015-05-19 Thread Andrii Biletskyi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2195: Status: Patch Available (was: Open) Add versionId to AbstractRequest.getErrorResponse and

[jira] [Updated] (KAFKA-2199) Make signing artifacts optional, setting maven repository possible from command line

2015-05-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2199: --- Reviewer: Jun Rao Make signing artifacts optional, setting maven repository possible from command line

[jira] [Updated] (KAFKA-976) Order-Preserving Mirror Maker Testcase

2015-05-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-976: -- Status: Open (was: Patch Available) The current patch is too old. Order-Preserving Mirror Maker Testcase

Re: Review Request 34273: remove roundrobin identical topic constraint in consumer coordinator

2015-05-19 Thread Guozhang Wang
On May 18, 2015, 9:46 p.m., Guozhang Wang wrote: core/src/main/scala/kafka/coordinator/PartitionAssignor.scala, lines 87-89 https://reviews.apache.org/r/34273/diff/1/?file=961395#file961395line87 I feel this logic's irregular assignment result may make it a bit harder to reason

Re: jira backlogs

2015-05-19 Thread Joel Koshy
Looks like a bug - maybe browser caching. The previous jira/confluence integration allowed you to refresh the table on the page directly. I will ping infra. For now I just changed it to use the link view, which just takes you to the jira. On Tue, May 19, 2015 at 9:23 AM, Jun Rao j...@confluent.io

Re: Review Request 34170: Patch for KAFKA-2191

2015-05-19 Thread Dong Lin
On May 19, 2015, 5:14 p.m., Jay Kreps wrote: clients/src/main/java/org/apache/kafka/common/metrics/stats/SampledStat.java, line 46 https://reviews.apache.org/r/34170/diff/3/?file=958800#file958800line46 Not sure if I grok why we need this while loop. Take a look at the patch I

Re: Review Request 34170: Patch for KAFKA-2191

2015-05-19 Thread Dong Lin
On May 19, 2015, 5:14 p.m., Jay Kreps wrote: Oh sure. Let me look through your patch. Thank you. - Dong --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34170/#review84341

Re: jira backlogs

2015-05-19 Thread Neha Narkhede
Just got a chance to take a look at this, so it takes me directly to JIRA. However, I think I know what you had in mind for this, Joel and it seems very useful :-) On Tue, May 19, 2015 at 10:21 AM, Joel Koshy jjkosh...@gmail.com wrote: Looks like a bug - maybe browser caching. The previous

[jira] [Updated] (KAFKA-2203) Get gradle build to work with Java 8

2015-05-19 Thread Gaju Bhat (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaju Bhat updated KAFKA-2203: - Fix Version/s: 0.8.1.2 Affects Version/s: 0.8.1.1 Status: Patch Available (was:

[jira] [Updated] (KAFKA-1374) LogCleaner (compaction) does not support compressed topics

2015-05-19 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1374: - Status: In Progress (was: Patch Available) LogCleaner (compaction) does not support compressed

[jira] [Created] (KAFKA-2204) Dynamic Configuration via ZK

2015-05-19 Thread Aditya Auradkar (JIRA)
Aditya Auradkar created KAFKA-2204: -- Summary: Dynamic Configuration via ZK Key: KAFKA-2204 URL: https://issues.apache.org/jira/browse/KAFKA-2204 Project: Kafka Issue Type: New Feature

Re: jira backlogs

2015-05-19 Thread Ewen Cheslack-Postava
It showed cached data for me until I logged in, then it refreshed on the next view. Not very intuitive, but if you know you need to be logged in it's probably just an annoyance that it might show old, cached data. On Tue, May 19, 2015 at 10:29 AM, Neha Narkhede n...@confluent.io wrote: Just got

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Jiangjie Qin
Hey Jay, That is also a viable solution. I think the main purpose is to let user know how long they can block, which is important. I have some question over the proposal, though. Will user still need to send linger.ms? Will request timeout cover linger.ms as well? My concern of letting request

[jira] [Created] (KAFKA-2203) Get gradle build to work with Java 8

2015-05-19 Thread Gaju Bhat (JIRA)
Gaju Bhat created KAFKA-2203: Summary: Get gradle build to work with Java 8 Key: KAFKA-2203 URL: https://issues.apache.org/jira/browse/KAFKA-2203 Project: Kafka Issue Type: Bug

Notes from KIP Hangout, May 19, 2015

2015-05-19 Thread Ashish Singh
Hey Guys, Below are my notes from today’s hangout. Feel free to add/modify the content. KIP Hangout (May 19, 2015) KIP-12 (sasl/ssl authentication): status check - Gwen has posted patch for KAFKA-1928, network refactoring. - Needed for sasl changes. - Pretty close to completion. -

Re: Writing a client: Connection pooling

2015-05-19 Thread Guozhang Wang
Hi Warren, Now I get your point. Today the request ordering guarantee is enforced on the broker's socket server such that if once a request is read from one socket, the socket key is set to not readable until that request's response is written back to the socket. This behavior could be changed

Re: Review Request 34397: Patch for KAFKA-1374

2015-05-19 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34397/#review84390 --- About duplicating the compression logic, one way would be: 1. Add

[jira] [Updated] (KAFKA-2203) Get gradle build to work with Java 8

2015-05-19 Thread Gaju Bhat (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaju Bhat updated KAFKA-2203: - Attachment: 0001-Special-case-java-8-and-javadoc-handling.patch Get gradle build to work with Java 8

RE: [VOTE] KIP-21 Dynamic Configuration

2015-05-19 Thread Aditya Auradkar
Updating the discussion with the latest comments. 1. We discussed adding 2 new API's (AlterConfig and DescribeConfig). I'll update KIP-21 with details on these. 2. Discussed during the KIP hangout. We are in agreement. (1) has a dependency on KIP-4 being completed. Rest of the work in the KIP

Re: [VOTE] KIP-21 Dynamic Configuration

2015-05-19 Thread Andrii Biletskyi
Hi, Sorry I wasn't able to participate. I don't have objections about removing config changes from AlterTopic (as I understand both AddedConfig and DeletedConfig) - you are welcome to update the KIP page. Thanks, Andrii Biletskyi On Tue, May 19, 2015 at 11:40 PM, Aditya Auradkar

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-19 Thread Mayuresh Gharat
IMO, having 4 different timeouts makes it confusing for the user and it requires the client to understand the internals of kafka. We should have a single timeout from the users perspective and handle other timeouts internally like a batch timeout. Mayuresh On Tue, May 19, 2015 at 12:42 PM,

[jira] [Commented] (KAFKA-2160) DelayedOperationPurgatory should remove the pair in watchersForKey with empty watchers list

2015-05-19 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551206#comment-14551206 ] Guozhang Wang commented on KAFKA-2160: -- Ping [~junrao] for another look at this patch

[jira] [Updated] (KAFKA-2198) kafka-topics.sh exits with 0 status on failures

2015-05-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2198: --- Reviewer: Neha Narkhede kafka-topics.sh exits with 0 status on failures

Re: [Discussion] Using Client Requests and Responses in Server

2015-05-19 Thread Andrii Biletskyi
Guys, I've just uploaded the patch which aligns versionId in getErrorResponse. (https://issues.apache.org/jira/browse/KAFKA-2195) Would be great if someone could review it because it's a blocker for other requests including MetadataRequest which I have to update to V1 for KIP-4. Thanks, Andrii

[jira] [Updated] (KAFKA-1374) LogCleaner (compaction) does not support compressed topics

2015-05-19 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1374: - Reviewer: Guozhang Wang (was: Joel Koshy) LogCleaner (compaction) does not support compressed

[jira] [Updated] (KAFKA-1906) Default the Kafka log directory to a relative path within the Kafka installation directory

2015-05-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1906: --- Reviewer: Jay Kreps Default the Kafka log directory to a relative path within the Kafka installation

[jira] [Updated] (KAFKA-1907) ZkClient can block controlled shutdown indefinitely

2015-05-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1907: --- Reviewer: Jun Rao ZkClient can block controlled shutdown indefinitely

[jira] [Commented] (KAFKA-2201) Open file handle leak

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550716#comment-14550716 ] Manikumar Reddy commented on KAFKA-2201: On Linux machines, tuning system ulimits

[jira] [Comment Edited] (KAFKA-2201) Open file handle leak

2015-05-19 Thread Manikumar Reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550716#comment-14550716 ] Manikumar Reddy edited comment on KAFKA-2201 at 5/19/15 4:29 PM:

Review Request 34415: Patch for KAFKA-2195

2015-05-19 Thread Andrii Biletskyi
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34415/ --- Review request for kafka. Bugs: KAFKA-2195

[jira] [Updated] (KAFKA-2196) remove roundrobin identical topic constraint in consumer coordinator

2015-05-19 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2196: --- Reviewer: Joel Koshy remove roundrobin identical topic constraint in consumer coordinator

  1   2   >