Re: Review Request 23895: Patch for KAFKA-1419

2014-08-07 Thread Ivan Lyutov
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23895/ --- (Updated Aug. 7, 2014, 10:17 a.m.) Review request for kafka. Bugs:

[jira] [Updated] (KAFKA-1419) cross build for scala 2.11

2014-08-07 Thread Ivan Lyutov (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Lyutov updated KAFKA-1419: --- Attachment: KAFKA-1419_2014-08-07_10:17:34.patch cross build for scala 2.11

[jira] [Commented] (KAFKA-1419) cross build for scala 2.11

2014-08-07 Thread Ivan Lyutov (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14089103#comment-14089103 ] Ivan Lyutov commented on KAFKA-1419: Updated reviewboard

Re: Review Request 23895: Patch for KAFKA-1419

2014-08-07 Thread Ivan Lyutov
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23895/ --- (Updated Aug. 7, 2014, 10:52 a.m.) Review request for kafka. Bugs:

[jira] [Updated] (KAFKA-1419) cross build for scala 2.11

2014-08-07 Thread Ivan Lyutov (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Lyutov updated KAFKA-1419: --- Attachment: KAFKA-1419_2014-08-07_10:52:18.patch cross build for scala 2.11

[jira] [Commented] (KAFKA-1419) cross build for scala 2.11

2014-08-07 Thread Ivan Lyutov (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14089125#comment-14089125 ] Ivan Lyutov commented on KAFKA-1419: Updated reviewboard

Re: Issue in using kafka SimpleConsumer code

2014-08-07 Thread pankaj ojha
Hi Team, Can you please provide any information on my above problem? Thanks Regards, Pankaj Ojha On Wed, Aug 6, 2014 at 8:04 PM, pankaj ojha pankajojh...@gmail.com wrote: Hi Team, I have a requirement of reading real time data using kafka and write to cassandra. For this I am using

Re: Issue in using kafka SimpleConsumer code

2014-08-07 Thread Harsha
Hi Pankaj, Do you notice any errors in kafka logs when your consumer stops reading data. Attaching those logs would be helpful in finding the issue. -Harsha On Thu, Aug 7, 2014, at 06:12 AM, pankaj ojha wrote: Hi Team, Can you please provide any information

Re: Issue in using kafka SimpleConsumer code

2014-08-07 Thread pankaj ojha
Hi Harsha, Thank you for replying. There is no error in the kafka logs. The output I had pasted in my previous mail is from kafka logs only. But when I restart the kafka process after incrementing the offset by 1, then it runs fine. Is there any known issue with the SimpleConsumer code? Thanks,

Re: Review Request 24214: Patch for KAFKA-1374

2014-08-07 Thread Manikumar Reddy O
On Aug. 6, 2014, 4:29 a.m., Jun Rao wrote: core/src/main/scala/kafka/log/LogCleaner.scala, lines 500-506 https://reviews.apache.org/r/24214/diff/1/?file=649265#file649265line500 Could we use Compressor.putRecord? Then,we don't have to worry about the details of the message

[jira] [Closed] (KAFKA-1556) Sorry it's Just Test

2014-08-07 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede closed KAFKA-1556. Sorry it's Just Test Key: KAFKA-1556 URL:

[jira] [Resolved] (KAFKA-1556) Sorry it's Just Test

2014-08-07 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede resolved KAFKA-1556. -- Resolution: Incomplete Sorry it's Just Test Key:

Re: Issue in using kafka SimpleConsumer code

2014-08-07 Thread Harsha
I suspect kafka.common.OffsetOutOfRangeException happening for your consumer if incrementing offset + 1 works. Try grep the server.log for that exception. I am not sure of any known issues with simpleconsumer that might causing this. Also copy this thread to us...@kafka.apache.org . Other users

Re: Review Request 24214: Patch for KAFKA-1374

2014-08-07 Thread Jun Rao
On Aug. 6, 2014, 4:29 a.m., Jun Rao wrote: core/src/main/scala/kafka/log/LogCleaner.scala, lines 500-506 https://reviews.apache.org/r/24214/diff/1/?file=649265#file649265line500 Could we use Compressor.putRecord? Then,we don't have to worry about the details of the message

Re: Review Request 23895: Patch for KAFKA-1419

2014-08-07 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23895/#review49920 --- Thanks for the patch. I think it's almost ready to go. A couple of

[jira] [Created] (KAFKA-1576) Make deleteTopic a bit more user friendly

2014-08-07 Thread Gwen Shapira (JIRA)
Gwen Shapira created KAFKA-1576: --- Summary: Make deleteTopic a bit more user friendly Key: KAFKA-1576 URL: https://issues.apache.org/jira/browse/KAFKA-1576 Project: Kafka Issue Type:

[jira] Subscription: outstanding kafka patches

2014-08-07 Thread jira
Issue Subscription Filter: outstanding kafka patches (117 issues) The list of outstanding kafka patches Subscriber: kafka-mailing-list Key Summary KAFKA-1569 Create tool to test correctness of transactions end-to-end https://issues.apache.org/jira/browse/KAFKA-1569 KAFKA-1567

[jira] [Created] (KAFKA-1577) Exception in ConnectionQuotas while shutting down

2014-08-07 Thread Joel Koshy (JIRA)
Joel Koshy created KAFKA-1577: - Summary: Exception in ConnectionQuotas while shutting down Key: KAFKA-1577 URL: https://issues.apache.org/jira/browse/KAFKA-1577 Project: Kafka Issue Type: Bug

Re: Uniform Distribution of Messages for Topic Across Partitions Without Effecting Performance

2014-08-07 Thread Bhavesh Mistry
The root of problem is consumer lag on one or two partition even with no op ( read log and discard it) consumer . Our use case is very simple. Send all the log lines to Brokers. But under storm of data (due to exception or application error etc), one or two partition gets lags behind while

[jira] [Commented] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in unit tests

2014-08-07 Thread Jonathan Natkins (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14089850#comment-14089850 ] Jonathan Natkins commented on KAFKA-1420: - Wanted to try to make sure this doesn't

[jira] [Commented] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in unit tests

2014-08-07 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14089867#comment-14089867 ] Guozhang Wang commented on KAFKA-1420: -- Sorry Jonathan, was completely swamped the

Re: Review Request 24196: Patch for KAFKA-1567

2014-08-07 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24196/ --- (Updated Aug. 7, 2014, 9:26 p.m.) Review request for kafka. Bugs: KAFKA-1567

[jira] [Updated] (KAFKA-1567) Metric memory leaking after closing the clients

2014-08-07 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-1567: Attachment: KAFKA-1567_2014-08-07_14:26:04.patch Metric memory leaking after closing the clients

[jira] [Commented] (KAFKA-1567) Metric memory leaking after closing the clients

2014-08-07 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14089879#comment-14089879 ] Jiangjie Qin commented on KAFKA-1567: - Updated reviewboard

Re: [New Feature Request] Ability to Inject Queue Implementation Async Mode

2014-08-07 Thread Bhavesh Mistry
Basically, requirement is to support message dropping policy in event when queue is full. When you get storm of data (in our case logging due to buggy application code), we would like to retain current message instead of first one in queue. We will mitigate this by rate limiting on producer

[jira] [Created] (KAFKA-1578) Controller should de-register all listeners upon designation

2014-08-07 Thread Guozhang Wang (JIRA)
Guozhang Wang created KAFKA-1578: Summary: Controller should de-register all listeners upon designation Key: KAFKA-1578 URL: https://issues.apache.org/jira/browse/KAFKA-1578 Project: Kafka

[jira] [Updated] (KAFKA-1578) Controller should de-register all listeners upon designation

2014-08-07 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1578: - Attachment: KAFKA-1578.patch Controller should de-register all listeners upon designation

Review Request 24480: Fix KAFKA-1578

2014-08-07 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24480/ --- Review request for kafka. Bugs: KAFKA-1578

[jira] [Commented] (KAFKA-1578) Controller should de-register all listeners upon designation

2014-08-07 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090017#comment-14090017 ] Guozhang Wang commented on KAFKA-1578: -- Created reviewboard

[jira] [Assigned] (KAFKA-1577) Exception in ConnectionQuotas while shutting down

2014-08-07 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani reassigned KAFKA-1577: - Assignee: Sriharsha Chintalapani Exception in ConnectionQuotas while

Re: Uniform Distribution of Messages for Topic Across Partitions Without Effecting Performance

2014-08-07 Thread Jun Rao
In the new producer, a client can specify the partition number for each message. Then, any partitioning strategy can be implemented by the client. Thanks, Jun On Thu, Aug 7, 2014 at 1:37 PM, Bhavesh Mistry mistry.p.bhav...@gmail.com wrote: The root of problem is consumer lag on one or two

Re: Review Request 24480: Fix KAFKA-1578

2014-08-07 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24480/#review50001 --- Thanks for the patch. Should we unregister the BrokerChangeListener