Re: confirm subscribe to dev@kafka.apache.org

2014-11-24 Thread Ivan Dyachkov
On Mon, 24 Nov 2014, at 10:23, dev-h...@kafka.apache.org wrote: Hi! This is the ezmlm program. I'm managing the dev@kafka.apache.org mailing list. I'm working for my owner, who can be reached at dev-ow...@kafka.apache.org. To confirm that you would like d...@dyachkov.org added

Re: Review Request 25995: Patch for KAFKA-1650

2014-11-24 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25995/ --- (Updated Nov. 24, 2014, 4:15 p.m.) Review request for kafka. Bugs:

[jira] [Updated] (KAFKA-1650) Mirror Maker could lose data on unclean shutdown.

2014-11-24 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-1650: Attachment: KAFKA-1650_2014-11-24_08:15:17.patch Mirror Maker could lose data on unclean shutdown.

[jira] [Commented] (KAFKA-1650) Mirror Maker could lose data on unclean shutdown.

2014-11-24 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223099#comment-14223099 ] Jiangjie Qin commented on KAFKA-1650: - Updated reviewboard

[jira] [Comment Edited] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Bhavesh Mistry (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223161#comment-14223161 ] Bhavesh Mistry edited comment on KAFKA-1642 at 11/24/14 5:26 PM:

[jira] [Commented] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Bhavesh Mistry (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223161#comment-14223161 ] Bhavesh Mistry commented on KAFKA-1642: --- [~ewencp], The way to reproduce this is to

[jira] [Comment Edited] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Bhavesh Mistry (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223161#comment-14223161 ] Bhavesh Mistry edited comment on KAFKA-1642 at 11/24/14 5:27 PM:

Re: [DISCUSSION] Message Metadata

2014-11-24 Thread Guozhang Wang
My bad, the link should be this: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Enriched+Message+Metadata On Fri, Nov 21, 2014 at 5:29 PM, Timothy Chen tnac...@gmail.com wrote: Hi Guozhang, I don't think that is publically accessible, can you update it to the Kafka wiki? Tim On

Re: Review Request 27391: Fix KAFKA-1634

2014-11-24 Thread Joel Koshy
On Nov. 21, 2014, 2:41 p.m., Joel Koshy wrote: core/src/main/scala/kafka/common/OffsetMetadataAndError.scala, line 17 https://reviews.apache.org/r/27391/diff/4/?file=766684#file766684line17 I thought we would be going with separate format for on-disk storage? E.g., one

[jira] [Comment Edited] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Bhavesh Mistry (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223161#comment-14223161 ] Bhavesh Mistry edited comment on KAFKA-1642 at 11/24/14 6:57 PM:

Re: Review Request 28268: Bump up default scala version from 2.10.1 to 2.10.4

2014-11-24 Thread Guozhang Wang
On Nov. 22, 2014, 2:24 a.m., Joel Koshy wrote: Looks good, although the patch does not apply - can you rebase; check that it builds properly and then we can close this out. This is a little wired.. both you and Gwen said it does not apply but when I apply the patch (patch -p1 --dry-run

[jira] [Updated] (KAFKA-1624) building on JDK 8 fails

2014-11-24 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1624: - Attachment: KAFKA-1624_2014-11-24_11:01:56.patch building on JDK 8 fails

Re: Review Request 28268: Fix KAFKA-1624

2014-11-24 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28268/ --- (Updated Nov. 24, 2014, 7:02 p.m.) Review request for kafka. Summary

[jira] [Commented] (KAFKA-1624) building on JDK 8 fails

2014-11-24 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223304#comment-14223304 ] Guozhang Wang commented on KAFKA-1624: -- Updated reviewboard

Re: Review Request 28268: Fix KAFKA-1624

2014-11-24 Thread Guozhang Wang
On Nov. 22, 2014, 2:24 a.m., Joel Koshy wrote: Looks good, although the patch does not apply - can you rebase; check that it builds properly and then we can close this out. Guozhang Wang wrote: This is a little wired.. both you and Gwen said it does not apply but when I apply the

[jira] [Updated] (KAFKA-1624) building on JDK 8 fails

2014-11-24 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1624: - Resolution: Fixed Status: Resolved (was: Patch Available) building on JDK 8 fails

[jira] [Commented] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Bhavesh Mistry (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223532#comment-14223532 ] Bhavesh Mistry commented on KAFKA-1642: --- Also Regarding KafkaProder.close() method

[jira] [Comment Edited] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Bhavesh Mistry (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223532#comment-14223532 ] Bhavesh Mistry edited comment on KAFKA-1642 at 11/24/14 9:21 PM:

[jira] [Comment Edited] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Bhavesh Mistry (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223532#comment-14223532 ] Bhavesh Mistry edited comment on KAFKA-1642 at 11/24/14 9:22 PM:

[jira] [Commented] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Bhavesh Mistry (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223571#comment-14223571 ] Bhavesh Mistry commented on KAFKA-1642: --- Here is exact steps how to reproducer it

[jira] [Commented] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Bhavesh Mistry (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223626#comment-14223626 ] Bhavesh Mistry commented on KAFKA-1642: --- Also, there is issue in my last patch. I

[jira] [Comment Edited] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Bhavesh Mistry (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223626#comment-14223626 ] Bhavesh Mistry edited comment on KAFKA-1642 at 11/24/14 10:16 PM:

[jira] [Commented] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223755#comment-14223755 ] Ewen Cheslack-Postava commented on KAFKA-1642: -- Ok, so as I suspected, you

[jira] [Updated] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Bhavesh Mistry (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhavesh Mistry updated KAFKA-1642: -- Affects Version/s: 0.8.1.1 [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when

[jira] [Updated] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-1642: - Priority: Blocker (was: Major) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network

[jira] [Commented] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Bhavesh Mistry (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223779#comment-14223779 ] Bhavesh Mistry commented on KAFKA-1642: --- [~ewencp], Thanks for looking into this

Re: Review Request 28121: Patch for KAFKA-1780

2014-11-24 Thread Neha Narkhede
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28121/#review62909 --- core/src/main/scala/kafka/consumer/ConsumerIterator.scala

[jira] [Commented] (KAFKA-1780) Add peek()/poll() for ConsumerIterator

2014-11-24 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223830#comment-14223830 ] Neha Narkhede commented on KAFKA-1780: -- [~ewencp] Sorry for the late review. Posted

[jira] [Updated] (KAFKA-1780) Add peek()/poll() for ConsumerIterator

2014-11-24 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-1780: - Attachment: KAFKA-1780_2014-11-24_17:24:54.patch Add peek()/poll() for

Re: Review Request 28121: Patch for KAFKA-1780

2014-11-24 Thread Ewen Cheslack-Postava
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28121/ --- (Updated Nov. 25, 2014, 1:24 a.m.) Review request for kafka. Bugs:

[jira] [Commented] (KAFKA-1780) Add peek()/poll() for ConsumerIterator

2014-11-24 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223905#comment-14223905 ] Ewen Cheslack-Postava commented on KAFKA-1780: -- Updated reviewboard

Re: Review Request 28121: Patch for KAFKA-1780

2014-11-24 Thread Ewen Cheslack-Postava
On Nov. 25, 2014, 12:26 a.m., Neha Narkhede wrote: core/src/main/scala/kafka/consumer/ConsumerIterator.scala, line 30 https://reviews.apache.org/r/28121/diff/1/?file=765888#file765888line30 Since we're changing the behavior of ConsumerIterator, let's document it here. For

[jira] [Comment Edited] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Bhavesh Mistry (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223779#comment-14223779 ] Bhavesh Mistry edited comment on KAFKA-1642 at 11/25/14 1:31 AM:

Re: Review Request 28121: Patch for KAFKA-1780

2014-11-24 Thread Neha Narkhede
On Nov. 25, 2014, 12:26 a.m., Neha Narkhede wrote: core/src/main/scala/kafka/consumer/ConsumerIterator.scala, line 30 https://reviews.apache.org/r/28121/diff/1/?file=765888#file765888line30 Since we're changing the behavior of ConsumerIterator, let's document it here. For

[jira] [Created] (KAFKA-1797) add the serializer/deserializer api to the new java client

2014-11-24 Thread Jun Rao (JIRA)
Jun Rao created KAFKA-1797: -- Summary: add the serializer/deserializer api to the new java client Key: KAFKA-1797 URL: https://issues.apache.org/jira/browse/KAFKA-1797 Project: Kafka Issue Type:

[jira] [Updated] (KAFKA-1797) add the serializer/deserializer api to the new java client

2014-11-24 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1797: --- Status: Patch Available (was: Open) add the serializer/deserializer api to the new java client

[jira] [Updated] (KAFKA-1797) add the serializer/deserializer api to the new java client

2014-11-24 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-1797: --- Attachment: kafka-1797.patch add the serializer/deserializer api to the new java client

[jira] [Commented] (KAFKA-1797) add the serializer/deserializer api to the new java client

2014-11-24 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223927#comment-14223927 ] Jun Rao commented on KAFKA-1797: Created reviewboard

Review Request 28423: Patch for kafka-1797

2014-11-24 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28423/ --- Review request for kafka. Bugs: kafka-1797

[DISCUSSION] adding the serializer api back to the new java producer

2014-11-24 Thread Jun Rao
Hi, Everyone, I'd like to start a discussion on whether it makes sense to add the serializer api back to the new java producer. Currently, the new java producer takes a byte array for both the key and the value. While this api is simple, it pushes the serialization logic into the application.

Re: Review Request 27990: Patch for KAFKA-1751

2014-11-24 Thread Neha Narkhede
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27990/#review62922 --- In addition to ensuring that the reassignment command doesn't

Re: [DISCUSSION] adding the serializer api back to the new java producer

2014-11-24 Thread Jay Kreps
This is admittedly late in the release cycle to make a change. To add to Jun's description the motivation was that we felt it would be better to change that interface now rather than after the release if it needed to change. The motivation for wanting to make a change was the ability to really be

Re: Review Request 28121: Patch for KAFKA-1780

2014-11-24 Thread Neha Narkhede
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28121/#review62926 --- Ship it! - Neha Narkhede On Nov. 25, 2014, 1:24 a.m., Ewen

[jira] [Commented] (KAFKA-1780) Add peek()/poll() for ConsumerIterator

2014-11-24 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223949#comment-14223949 ] Neha Narkhede commented on KAFKA-1780: -- This patch touches the consumer iterator.

[jira] [Commented] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Bhavesh Mistry (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224039#comment-14224039 ] Bhavesh Mistry commented on KAFKA-1642: --- Here are some more cases to reproduce this

[jira] [Commented] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Bhavesh Mistry (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224041#comment-14224041 ] Bhavesh Mistry commented on KAFKA-1642: --- [~ewencp], I hope above steps will give

Re: [DISCUSSION] adding the serializer api back to the new java producer

2014-11-24 Thread Gwen Shapira
As one of the people who spent too much time building Avro repositories, +1 on bringing serializer API back. I think it will make the new producer easier to work with. Gwen On Mon, Nov 24, 2014 at 6:13 PM, Jay Kreps jay.kr...@gmail.com wrote: This is admittedly late in the release cycle to

[jira] [Comment Edited] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Bhavesh Mistry (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224041#comment-14224041 ] Bhavesh Mistry edited comment on KAFKA-1642 at 11/25/14 4:37 AM:

[jira] [Comment Edited] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Bhavesh Mistry (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224041#comment-14224041 ] Bhavesh Mistry edited comment on KAFKA-1642 at 11/25/14 4:39 AM:

[jira] [Comment Edited] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Bhavesh Mistry (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224041#comment-14224041 ] Bhavesh Mistry edited comment on KAFKA-1642 at 11/25/14 4:40 AM:

[jira] [Commented] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Bhavesh Mistry (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224046#comment-14224046 ] Bhavesh Mistry commented on KAFKA-1642: --- Also, Are you going to port back the back

[jira] [Comment Edited] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Bhavesh Mistry (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224046#comment-14224046 ] Bhavesh Mistry edited comment on KAFKA-1642 at 11/25/14 4:43 AM:

Re: [DISCUSSION] adding the serializer api back to the new java producer

2014-11-24 Thread Sriram Subramanian
Looked at the patch. +1 from me. On 11/24/14 8:29 PM, Gwen Shapira gshap...@cloudera.com wrote: As one of the people who spent too much time building Avro repositories, +1 on bringing serializer API back. I think it will make the new producer easier to work with. Gwen On Mon, Nov 24, 2014 at

[jira] [Comment Edited] (KAFKA-1642) [Java New Producer Kafka Trunk] CPU Usage Spike to 100% when network connection is lost

2014-11-24 Thread Bhavesh Mistry (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224041#comment-14224041 ] Bhavesh Mistry edited comment on KAFKA-1642 at 11/25/14 5:37 AM: