[jira] [Commented] (KAFKA-2176) DefaultPartitioner doesn't perform consistent hashing based on

2015-05-12 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539320#comment-14539320 ] Gwen Shapira commented on KAFKA-2176: - Do we want to fix this? It is not an issue in

[jira] [Comment Edited] (KAFKA-2176) DefaultPartitioner doesn't perform consistent hashing based on

2015-05-12 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539320#comment-14539320 ] Gwen Shapira edited comment on KAFKA-2176 at 5/12/15 6:11 AM: --

[jira] [Commented] (KAFKA-2176) DefaultPartitioner doesn't perform consistent hashing based on

2015-05-12 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539393#comment-14539393 ] Igor Maravić commented on KAFKA-2176: - I don't see why this wouldn't be fixed. As far

[jira] [Commented] (KAFKA-2176) DefaultPartitioner doesn't perform consistent hashing based on

2015-05-12 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539409#comment-14539409 ] Gwen Shapira commented on KAFKA-2176: - 0.8.2 has new.producer parameter that can be

[jira] [Issue Comment Deleted] (KAFKA-2176) DefaultPartitioner doesn't perform consistent hashing based on

2015-05-12 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Maravić updated KAFKA-2176: Comment: was deleted (was: Didn't know that. How much is this battle tested? We had a lot of

[jira] [Commented] (KAFKA-2176) DefaultPartitioner doesn't perform consistent hashing based on

2015-05-12 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539430#comment-14539430 ] Igor Maravić commented on KAFKA-2176: - Didn't know that. How much is this battle

[jira] [Commented] (KAFKA-2176) DefaultPartitioner doesn't perform consistent hashing based on

2015-05-12 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539429#comment-14539429 ] Igor Maravić commented on KAFKA-2176: - Didn't know that. How much is this battle

[jira] [Commented] (KAFKA-2146) adding partition did not find the correct startIndex

2015-05-12 Thread chenshangan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539476#comment-14539476 ] chenshangan commented on KAFKA-2146: [~guozhang] It's ok. For #2 I totally agree,

Re: Review Request 33065: Patch for 1928

2015-05-12 Thread Gwen Shapira
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33065/ --- (Updated May 12, 2015, 8:40 a.m.) Review request for kafka. Summary

[jira] [Created] (KAFKA-2187) Introduce merge-kafka-pr.py script

2015-05-12 Thread Ismael Juma (JIRA)
Ismael Juma created KAFKA-2187: -- Summary: Introduce merge-kafka-pr.py script Key: KAFKA-2187 URL: https://issues.apache.org/jira/browse/KAFKA-2187 Project: Kafka Issue Type: New Feature

[jira] [Commented] (KAFKA-2176) DefaultPartitioner doesn't perform consistent hashing based on

2015-05-12 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539451#comment-14539451 ] Gwen Shapira commented on KAFKA-2176: - 1. Regarding battle testing, LinkedIn is the

Re: [DISCUSS] KIP-21 Configuration Management

2015-05-12 Thread Jay Kreps
Hmm, here is how I think we can change the split brain proposal to make it a bit better: 1. Get rid of broker overrides, this is just done in the config file. This makes the precedence chain a lot clearer (e.g. zk always overrides file on a per-entity basis). 2. Get rid of the notion of dynamic

Re: Review Request 34050: Patch for KAFKA-2169

2015-05-12 Thread Parth Brahmbhatt
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34050/#review83427 --- core/src/main/scala/kafka/controller/KafkaController.scala

[jira] [Commented] (KAFKA-2189) Snappy compression of message batches less efficient in 0.8.2.1

2015-05-12 Thread Olson,Andrew (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540351#comment-14540351 ] Olson,Andrew commented on KAFKA-2189: - I have verified that the issue [1] was

[jira] [Commented] (KAFKA-2147) Unbalanced replication can cause extreme purgatory growth

2015-05-12 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540168#comment-14540168 ] Jun Rao commented on KAFKA-2147: A few other questions. 1. What's the produce request

[jira] [Commented] (KAFKA-2147) Unbalanced replication can cause extreme purgatory growth

2015-05-12 Thread Evan Huus (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540218#comment-14540218 ] Evan Huus commented on KAFKA-2147: -- 1. 4-8k messages/s snappy-compressed, taking 2.5-5

[jira] [Commented] (KAFKA-1554) Corrupt index found on clean startup

2015-05-12 Thread Mayuresh Gharat (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540270#comment-14540270 ] Mayuresh Gharat commented on KAFKA-1554: This sounds interesting. We may try to

[jira] [Created] (KAFKA-2189) Snappy compression of message batches less efficient in 0.8.2.1

2015-05-12 Thread Olson,Andrew (JIRA)
Olson,Andrew created KAFKA-2189: --- Summary: Snappy compression of message batches less efficient in 0.8.2.1 Key: KAFKA-2189 URL: https://issues.apache.org/jira/browse/KAFKA-2189 Project: Kafka

[jira] [Commented] (KAFKA-1554) Corrupt index found on clean startup

2015-05-12 Thread Steve Miller (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540520#comment-14540520 ] Steve Miller commented on KAFKA-1554: - The foo-n.old thing was just a way I knew I

[jira] [Commented] (KAFKA-1554) Corrupt index found on clean startup

2015-05-12 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540494#comment-14540494 ] Joel Koshy commented on KAFKA-1554: --- Thanks Steve - sorry I misread your earlier

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-12 Thread Ewen Cheslack-Postava
I think my confusion is coming from this: So in this KIP, we only address (3). The only public interface change is a new configuration of request timeout (and maybe change the configuration name of TIMEOUT_CONFIG to REPLICATION_TIMEOUT_CONFIG). There are 3 possible compatibility issues I see

Re: [DISCUSS] KIP-21 Configuration Management

2015-05-12 Thread Joel Koshy
The lack of audit could be addressed to some degree by an internal __config_changes topic which can have very long retention. Also, per the hangout summary that Gwen sent out it appears that we decided against supporting SIGHUP/dynamic configs for the broker. Thanks, Joel On Tue, May 12, 2015

Re: Review Request 33378: Patch for KAFKA-2136

2015-05-12 Thread Dong Lin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33378/#review83439 --- core/src/main/scala/kafka/server/ReplicaManager.scala

Re: Review Request 33049: Patch for KAFKA-2084

2015-05-12 Thread Dong Lin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33049/#review83441 --- clients/src/main/java/org/apache/kafka/common/metrics/Sensor.java

[jira] [Commented] (KAFKA-2147) Unbalanced replication can cause extreme purgatory growth

2015-05-12 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540596#comment-14540596 ] Jun Rao commented on KAFKA-2147: 1. How many producer requests/sec are there on each

Re: Review Request 31850: Patch for KAFKA-1660

2015-05-12 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31850/#review83433 --- Ship it! Minor comments - I will address these on check-in.

Re: Review Request 33049: Patch for KAFKA-2084

2015-05-12 Thread Aditya Auradkar
On May 12, 2015, 7:38 p.m., Dong Lin wrote: clients/src/main/java/org/apache/kafka/common/metrics/Sensor.java, line 117 https://reviews.apache.org/r/33049/diff/9/?file=955824#file955824line117 metric.value(timeMs), which translates to Rate.measure(config, timeMs), may return

Re: Review Request 33378: Patch for KAFKA-2136

2015-05-12 Thread Aditya Auradkar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33378/ --- (Updated May 12, 2015, 9:40 p.m.) Review request for kafka, Joel Koshy and Jun

Re: Review Request 34070: Patch for KAFKA-2186

2015-05-12 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34070/#review83472 --- Ship it! Ship It! - Joel Koshy On May 12, 2015, 1:39 a.m.,

[jira] [Commented] (KAFKA-2167) ZkUtils updateEphemeralPath JavaDoc (spelling and correctness)

2015-05-12 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540779#comment-14540779 ] ASF GitHub Bot commented on KAFKA-2167: --- GitHub user nssalian opened a pull request:

[GitHub] kafka pull request: Kafka 2167

2015-05-12 Thread nssalian
GitHub user nssalian opened a pull request: https://github.com/apache/kafka/pull/64 Kafka 2167 Changed ZkUtils.scala for KAFKA-2167 You can merge this pull request into a Git repository by running: $ git pull https://github.com/nssalian/kafka KAFKA-2167 Alternatively you can

[jira] [Commented] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.

2015-05-12 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540793#comment-14540793 ] Jiangjie Qin commented on KAFKA-1660: - Updated reviewboard

Re: [KIP-DISCUSSION] KIP-22 Expose a Partitioner interface in the new producer

2015-05-12 Thread Jiangjie Qin
Hey Harsha, It looks you created the KIP page at wrong place. . . Can you move the page to a child page of https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposa ls Thanks. Jiangjie (Becket) Qin On 5/6/15, 6:12 PM, Harsha ka...@harsha.io wrote: Thanks for the review Joel.

[jira] [Updated] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.

2015-05-12 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-1660: Attachment: KAFKA-1660_2015-05-12_14:28:53.patch Ability to call close() with a timeout on the

[jira] [Commented] (KAFKA-2136) Client side protocol changes to return quota delays

2015-05-12 Thread Aditya A Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540818#comment-14540818 ] Aditya A Auradkar commented on KAFKA-2136: -- Updated reviewboard

[jira] [Commented] (KAFKA-2150) FetcherThread backoff need to grab lock before wait on condition.

2015-05-12 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540651#comment-14540651 ] Sriharsha Chintalapani commented on KAFKA-2150: --- Thanks for merging it

Re: Review Request 34070: Patch for KAFKA-2186

2015-05-12 Thread Joel Koshy
On May 12, 2015, 2:31 a.m., Aditya Auradkar wrote: core/src/main/scala/kafka/javaapi/consumer/ConsumerConnector.java, line 73 https://reviews.apache.org/r/34070/diff/1/?file=955930#file955930line73 How does this work if the consumer doesn't own these partitions? Is it possible to

[GitHub] kafka pull request: Corrected the Changes in ZkUtils.scala - KAFKA...

2015-05-12 Thread nssalian
Github user nssalian closed the pull request at: https://github.com/apache/kafka/pull/63 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-2167) ZkUtils updateEphemeralPath JavaDoc (spelling and correctness)

2015-05-12 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540782#comment-14540782 ] ASF GitHub Bot commented on KAFKA-2167: --- Github user nssalian closed the pull

[jira] [Commented] (KAFKA-2176) DefaultPartitioner doesn't perform consistent hashing based on

2015-05-12 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540784#comment-14540784 ] Jiangjie Qin commented on KAFKA-2176: - [~i_maravic], LinkedIn uses the mirror maker on

[jira] [Updated] (KAFKA-2136) Client side protocol changes to return quota delays

2015-05-12 Thread Aditya A Auradkar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya A Auradkar updated KAFKA-2136: - Attachment: KAFKA-2136_2015-05-12_14:40:44.patch Client side protocol changes to return

Re: Review Request 33378: Patch for KAFKA-2136

2015-05-12 Thread Aditya Auradkar
On May 12, 2015, 7:39 p.m., Dong Lin wrote: core/src/main/scala/kafka/server/ReplicaManager.scala, line 428 https://reviews.apache.org/r/33378/diff/4/?file=955709#file955709line428 Will readFromLocalLog() read data into memory before the cilent's quota is checked? It shouldn't.

[jira] [Comment Edited] (KAFKA-2147) Unbalanced replication can cause extreme purgatory growth

2015-05-12 Thread Evan Huus (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540669#comment-14540669 ] Evan Huus edited comment on KAFKA-2147 at 5/12/15 8:43 PM: --- 1.

[jira] [Commented] (KAFKA-2147) Unbalanced replication can cause extreme purgatory growth

2015-05-12 Thread Evan Huus (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540669#comment-14540669 ] Evan Huus commented on KAFKA-2147: -- 1. 150-200 base on a quick wireshark capture 2.

[jira] [Commented] (KAFKA-2186) Follow-up patch of KAFKA-1650

2015-05-12 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540706#comment-14540706 ] Joel Koshy commented on KAFKA-2186: --- Hmm.. though this is trivial and an omission from

Re: Review Request 31850: Patch for KAFKA-1660

2015-05-12 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31850/ --- (Updated May 12, 2015, 9:29 p.m.) Review request for kafka. Bugs: KAFKA-1660

[jira] [Updated] (KAFKA-2189) Snappy compression of message batches less efficient in 0.8.2.1

2015-05-12 Thread Olson,Andrew (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olson,Andrew updated KAFKA-2189: Component/s: compression build Snappy compression of message batches less

Re: Review Request 33378: Patch for KAFKA-2136

2015-05-12 Thread Aditya Auradkar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33378/ --- (Updated May 12, 2015, 9:42 p.m.) Review request for kafka, Joel Koshy and Jun

[jira] [Created] (KAFKA-2190) Incorporate close(timeout) to Mirror Maker

2015-05-12 Thread Jiangjie Qin (JIRA)
Jiangjie Qin created KAFKA-2190: --- Summary: Incorporate close(timeout) to Mirror Maker Key: KAFKA-2190 URL: https://issues.apache.org/jira/browse/KAFKA-2190 Project: Kafka Issue Type: Bug

[jira] [Commented] (KAFKA-1347) Create a system test for network partitions

2015-05-12 Thread Geoffrey Anderson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541125#comment-14541125 ] Geoffrey Anderson commented on KAFKA-1347: -- Just making sure you're aware of work

[jira] [Commented] (KAFKA-1604) System Test for Transaction Management

2015-05-12 Thread Geoffrey Anderson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541123#comment-14541123 ] Geoffrey Anderson commented on KAFKA-1604: -- Just making sure you're aware of work

[jira] [Commented] (KAFKA-2003) Add upgrade tests

2015-05-12 Thread Geoffrey Anderson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541121#comment-14541121 ] Geoffrey Anderson commented on KAFKA-2003: -- Just making sure you're aware of work

[jira] [Commented] (KAFKA-1918) System test for ZooKeeper quorum failure scenarios

2015-05-12 Thread Geoffrey Anderson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541127#comment-14541127 ] Geoffrey Anderson commented on KAFKA-1918: -- Just making sure you're aware of work

[jira] [Commented] (KAFKA-976) Order-Preserving Mirror Maker Testcase

2015-05-12 Thread Geoffrey Anderson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541124#comment-14541124 ] Geoffrey Anderson commented on KAFKA-976: - Just making sure you're aware of work

[jira] [Commented] (KAFKA-1904) run sanity failed test

2015-05-12 Thread Geoffrey Anderson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541120#comment-14541120 ] Geoffrey Anderson commented on KAFKA-1904: -- Just making sure you're aware of work

[jira] [Commented] (KAFKA-1898) compatibility testing framework

2015-05-12 Thread Geoffrey Anderson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541122#comment-14541122 ] Geoffrey Anderson commented on KAFKA-1898: -- Just making sure you're aware of work

[jira] [Commented] (KAFKA-1888) Add a rolling upgrade system test

2015-05-12 Thread Geoffrey Anderson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541118#comment-14541118 ] Geoffrey Anderson commented on KAFKA-1888: -- Just making sure you're aware of work

Re: Review Request 34144: Patch for KAFKA-2189

2015-05-12 Thread Ismael Juma
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34144/ --- (Updated May 13, 2015, 12:59 a.m.) Review request for kafka. Bugs:

[jira] [Updated] (KAFKA-2189) Snappy compression of message batches less efficient in 0.8.2.1

2015-05-12 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-2189: --- Attachment: KAFKA-2189.patch Snappy compression of message batches less efficient in 0.8.2.1

[jira] [Commented] (KAFKA-2189) Snappy compression of message batches less efficient in 0.8.2.1

2015-05-12 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541106#comment-14541106 ] Ismael Juma commented on KAFKA-2189: Created reviewboard

Review Request 34144: Patch for KAFKA-2189

2015-05-12 Thread Ismael Juma
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34144/ --- Review request for kafka. Bugs: KAFKA-2189

[jira] [Updated] (KAFKA-2189) Snappy compression of message batches less efficient in 0.8.2.1

2015-05-12 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-2189: --- Assignee: Ismael Juma Status: Patch Available (was: Open) Snappy compression of message

[jira] [Commented] (KAFKA-2171) System Test for Quotas

2015-05-12 Thread Geoffrey Anderson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541116#comment-14541116 ] Geoffrey Anderson commented on KAFKA-2171: -- Just pinging this thread to make sure

Re: Review Request 33065: Patch for KAFKA-1928

2015-05-12 Thread Gwen Shapira
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33065/ --- (Updated May 12, 2015, 9 a.m.) Review request for kafka. Summary (updated)

[jira] [Updated] (KAFKA-1928) Move kafka.network over to using the network classes in org.apache.kafka.common.network

2015-05-12 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1928: Attachment: KAFKA-1928_2015-05-12_12:00:37.patch Move kafka.network over to using the network

Re: Review Request 33065: Patch for KAFKA-1928

2015-05-12 Thread Gwen Shapira
On May 6, 2015, 10:50 p.m., Jun Rao wrote: clients/src/main/java/org/apache/kafka/common/network/ByteBufferSend.java, lines 55-64 https://reviews.apache.org/r/33065/diff/4/?file=947526#file947526line55 Since we always use a 4 byte size, perhaps we should make both remaining()

Re: Review Request 33065: Patch for KAFKA-1928

2015-05-12 Thread Gwen Shapira
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33065/ --- (Updated May 12, 2015, 9:58 a.m.) Review request for kafka. Bugs: 1928 and

[jira] [Updated] (KAFKA-1928) Move kafka.network over to using the network classes in org.apache.kafka.common.network

2015-05-12 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1928: Attachment: KAFKA-1928_2015-05-12_12:57:57.patch Move kafka.network over to using the network

[jira] [Commented] (KAFKA-1928) Move kafka.network over to using the network classes in org.apache.kafka.common.network

2015-05-12 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539606#comment-14539606 ] Gwen Shapira commented on KAFKA-1928: - Updated reviewboard

[jira] [Updated] (KAFKA-2175) Reduce server log verbosity at info level

2015-05-12 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-2175: -- Resolution: Pending Closed Status: Resolved (was: Patch Available) +1 Thanks for the patch -

Re: [KIP-DISCUSSION] KIP-22 Expose a Partitioner interface in the new producer

2015-05-12 Thread Jiangjie Qin
Hi Harsha, If you open this link https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposa ls All the KIPs are the child page of this page which you can see from the left bar. Only KIP-22 is missing. It looks you created it as a child page of

Re: [DISCUSS] KIP-19 Add a request timeout to NetworkClient

2015-05-12 Thread Mayuresh Gharat
+1 Becket. That would give enough time for clients to move. We should make this change very clear. Thanks, Mayuresh On Tue, May 12, 2015 at 1:45 PM, Jiangjie Qin j...@linkedin.com.invalid wrote: Hey Ewen, Very good summary about the compatibility. What you proposed makes sense. So

[jira] [Updated] (KAFKA-2147) Unbalanced replication can cause extreme purgatory growth

2015-05-12 Thread Evan Huus (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Evan Huus updated KAFKA-2147: - Attachment: watch-lists.log watch list purging logs Unbalanced replication can cause extreme purgatory

Build failed in Jenkins: KafkaPreCommit #100

2015-05-12 Thread Apache Jenkins Server
See https://builds.apache.org/job/KafkaPreCommit/100/changes Changes: [jjkoshy] KAFKA-2175; Change INFO to DEBUG for socket close on no error; and partition reassignment JSON; reviewed by Gwen Shapira and Joel Koshy [jjkoshy] KAFKA-2163; The offsets manager's stale-offset-cleanup and offset

Add missing API to old high level consumer

2015-05-12 Thread Jiangjie Qin
Hi, I just noticed that in KAFKA-1650 (which is before we use KIP) we added an offset commit method in high level consumer that commits offsets using a user provided offset map. public void commitOffsets(MapTopicPartition, OffsetAndMetadata offsetsToCommit, boolean retryOnFailure); This

[jira] [Commented] (KAFKA-2147) Unbalanced replication can cause extreme purgatory growth

2015-05-12 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540917#comment-14540917 ] Jun Rao commented on KAFKA-2147: Created reviewboard

Review Request 34125: Patch for KAFKA-2147

2015-05-12 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34125/ --- Review request for kafka. Bugs: KAFKA-2147

[jira] [Updated] (KAFKA-2189) Snappy compression of message batches less efficient in 0.8.2.1

2015-05-12 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2189: --- Priority: Blocker (was: Major) Fix Version/s: 0.8.3 Assignee: (was: Jay Kreps)

[jira] [Commented] (KAFKA-1690) new java producer needs ssl support as a client

2015-05-12 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540989#comment-14540989 ] Sriharsha Chintalapani commented on KAFKA-1690: --- Updated reviewboard

[jira] [Updated] (KAFKA-1690) new java producer needs ssl support as a client

2015-05-12 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani updated KAFKA-1690: -- Attachment: KAFKA-1690_2015-05-12_16:20:08.patch new java producer needs ssl

Re: [KIP-DISCUSSION] KIP-22 Expose a Partitioner interface in the new producer

2015-05-12 Thread Jiangjie Qin
Hey Harsha, The link in your last email gives page not found. . . Jiangjie (Becket) Qin On 5/12/15, 3:12 PM, Sriharsha Chintalapani ka...@harsha.io wrote: Hi Jiangjie, Its under https://cwiki.apache.org/confluence/display/KAFKA/KIP-+22++Expose+a+Partit

Re: [KIP-DISCUSSION] KIP-22 Expose a Partitioner interface in the new producer

2015-05-12 Thread Sriharsha Chintalapani
Hi Jiangjie,        Its under  https://cwiki.apache.org/confluence/display/KAFKA/KIP-+22++Expose+a+Partitioner+interface+in+the+new+producer I checked other KIPS they are under /KAFKA  as well. Thanks, Harsha On May 12, 2015 at 2:12:30 PM, Jiangjie Qin (j...@linkedin.com.invalid) wrote: Hey

[jira] [Commented] (KAFKA-2147) Unbalanced replication can cause extreme purgatory growth

2015-05-12 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540921#comment-14540921 ] Jun Rao commented on KAFKA-2147: Looking a bit more at our code, it seems that this could

[jira] [Updated] (KAFKA-1660) Ability to call close() with a timeout on the Java Kafka Producer.

2015-05-12 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1660: -- Resolution: Pending Closed Status: Resolved (was: Patch Available) Thanks for the patch -

[jira] [Updated] (KAFKA-2163) Offsets manager cache should prevent stale-offset-cleanup while an offset load is in progress; otherwise we can lose consumer offsets

2015-05-12 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-2163: -- Resolution: Pending Closed Status: Resolved (was: Patch Available) Committed to trunk.

[jira] [Updated] (KAFKA-2147) Unbalanced replication can cause extreme purgatory growth

2015-05-12 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2147: --- Assignee: Jun Rao (was: Joel Koshy) Status: Patch Available (was: Open) Unbalanced replication can

[jira] [Updated] (KAFKA-2147) Unbalanced replication can cause extreme purgatory growth

2015-05-12 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2147: --- Attachment: KAFKA-2147.patch Unbalanced replication can cause extreme purgatory growth

Re: Review Request 33620: Patch for KAFKA-1690

2015-05-12 Thread Sriharsha Chintalapani
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33620/ --- (Updated May 12, 2015, 11:20 p.m.) Review request for kafka. Bugs:

[jira] [Created] (KAFKA-2188) JBOD Support

2015-05-12 Thread Andrii Biletskyi (JIRA)
Andrii Biletskyi created KAFKA-2188: --- Summary: JBOD Support Key: KAFKA-2188 URL: https://issues.apache.org/jira/browse/KAFKA-2188 Project: Kafka Issue Type: Bug Reporter:

Review Request 34103: Patch for KAFKA-2188

2015-05-12 Thread Andrii Biletskyi
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34103/ --- Review request for kafka. Bugs: KAFKA-2188

[jira] [Commented] (KAFKA-2188) JBOD Support

2015-05-12 Thread Andrii Biletskyi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539777#comment-14539777 ] Andrii Biletskyi commented on KAFKA-2188: - Created reviewboard

[jira] [Updated] (KAFKA-2188) JBOD Support

2015-05-12 Thread Andrii Biletskyi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2188: Status: Patch Available (was: Open) JBOD Support Key:

[jira] [Updated] (KAFKA-2188) JBOD Support

2015-05-12 Thread Andrii Biletskyi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2188: Attachment: KAFKA-2188.patch JBOD Support Key: KAFKA-2188

Re: Message corruption with new Java client + snappy + broker restart

2015-05-12 Thread Jiangjie Qin
Does this related to KAFKA-2189? Jiangjie (Becket) Qin On 5/12/15, 9:41 PM, Roger Hoover roger.hoo...@gmail.com wrote: Hi, When using Samza 0.9.0 which uses the new Java producer client and snappy enabled, I see messages getting corrupted on the client side. It never happens with the old

Re: Review Request 33557: Patch for KAFKA-1936

2015-05-12 Thread Dong Lin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33557/ --- (Updated May 12, 2015, 2:32 p.m.) Review request for kafka. Bugs: KAFKA-1936

[jira] [Commented] (KAFKA-1936) Track offset commit requests separately from produce requests

2015-05-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539906#comment-14539906 ] Dong Lin commented on KAFKA-1936: - Updated reviewboard

[jira] [Updated] (KAFKA-1936) Track offset commit requests separately from produce requests

2015-05-12 Thread Dong Lin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin updated KAFKA-1936: Attachment: KAFKA-1936_2015-05-12_07:32:07.patch Track offset commit requests separately from produce

[jira] [Commented] (KAFKA-1554) Corrupt index found on clean startup

2015-05-12 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539883#comment-14539883 ] Joel Koshy commented on KAFKA-1554: --- That is useful to know - it might help reproducing

Re: Review Request 33557: Patch for KAFKA-1936

2015-05-12 Thread Dong Lin
On May 11, 2015, 5:25 p.m., Joel Koshy wrote: core/src/main/scala/kafka/server/KafkaRequestHandler.scala, line 147 https://reviews.apache.org/r/33557/diff/4/?file=949209#file949209line147 How about markBrokerTopicMeters or updateBrokerTopicStats I

Message corruption with new Java client + snappy + broker restart

2015-05-12 Thread Roger Hoover
Hi, When using Samza 0.9.0 which uses the new Java producer client and snappy enabled, I see messages getting corrupted on the client side. It never happens with the old producer and it never happens with lz4, gzip, or no compression. It only happens when a broker gets restarted (or maybe just

  1   2   >