[jira] [Commented] (KAFKA-2044) Support requests and responses from o.a.k.common in KafkaApis

2015-03-24 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378814#comment-14378814 ] Gwen Shapira commented on KAFKA-2044: - Created reviewboard

Jenkins build is back to normal : KafkaPreCommit #42

2015-03-24 Thread Apache Jenkins Server
See https://builds.apache.org/job/KafkaPreCommit/42/changes

KAFKA-2042

2015-03-24 Thread Guozhang Wang
Hello, We found a serious bug while testing flush() calls in the new producer, which is summarized in KAFKA-2042. In general, when the producer starts up it will try to refresh metadata with empty topic list, and hence get all the topic metadata. When sending the message with some topic later,

Re: [VOTE] KIP-15 add a close with timeout to new producer

2015-03-24 Thread Guozhang Wang
+1. On Tue, Mar 24, 2015 at 2:15 PM, Jiangjie Qin j...@linkedin.com.invalid wrote: https://cwiki.apache.org/confluence/display/KAFKA/KIP-15+-+Add+a+close+method+with+a+timeout+in+the+producer As a short summary, the new interface will be as following: Close(Long Timeout, TimeUnit timeUnit)

Re: [Discussion] Using Client Requests and Responses in Server

2015-03-24 Thread Gwen Shapira
OK, I posted a working patch on KAFKA-2044 and https://reviews.apache.org/r/32459/diff/. There are few decisions there than can be up to discussion (factory method on AbstractRequestResponse, the new handleErrors in request API), but as far as support for o.a.k.common requests in core goes, it

Re: Review Request 32434: Patch for KAFKA-2042

2015-03-24 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32434/#review77675 --- Ship it! Ship It! - Guozhang Wang On March 24, 2015, 8:57 p.m.,

[jira] [Updated] (KAFKA-2042) New producer metadata update always get all topics.

2015-03-24 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2042: - Fix Version/s: 0.8.3 New producer metadata update always get all topics.

[jira] [Commented] (KAFKA-2042) New producer metadata update always get all topics.

2015-03-24 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378828#comment-14378828 ] Guozhang Wang commented on KAFKA-2042: -- Thanks for the patch, committed to trunk.

[jira] [Commented] (KAFKA-2035) Add a topic config cache.

2015-03-24 Thread Parth Brahmbhatt (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378925#comment-14378925 ] Parth Brahmbhatt commented on KAFKA-2035: - Posted a review

[jira] [Updated] (KAFKA-2047) Accelarate consumer rebalance in mirror maker.

2015-03-24 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-2047: Attachment: KAFKA-2047.patch Accelarate consumer rebalance in mirror maker.

[jira] [Commented] (KAFKA-2047) Accelarate consumer rebalance in mirror maker.

2015-03-24 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378935#comment-14378935 ] Jiangjie Qin commented on KAFKA-2047: - Created reviewboard

[jira] [Updated] (KAFKA-2047) Accelarate consumer rebalance in mirror maker.

2015-03-24 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-2047: Status: Patch Available (was: Open) Accelarate consumer rebalance in mirror maker.

Review Request 32465: Patch for KAFKA-2047

2015-03-24 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32465/ --- Review request for kafka. Bugs: KAFKA-2047

[jira] [Updated] (KAFKA-2044) Support requests and responses from o.a.k.common in KafkaApis

2015-03-24 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-2044: Status: Patch Available (was: Open) Support requests and responses from o.a.k.common in KafkaApis

Review Request 32459: Patch for KAFKA-2044

2015-03-24 Thread Gwen Shapira
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32459/ --- Review request for kafka. Bugs: KAFKA-2044

[jira] [Updated] (KAFKA-2044) Support requests and responses from o.a.k.common in KafkaApis

2015-03-24 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-2044: Attachment: KAFKA-2044.patch Support requests and responses from o.a.k.common in KafkaApis

Re: Review Request 31366: Patch for KAFKA-1461

2015-03-24 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31366/#review77674 --- core/src/main/scala/kafka/server/AbstractFetcherThread.scala

[jira] [Commented] (KAFKA-2046) Delete topic still doesn't work

2015-03-24 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378842#comment-14378842 ] Joel Koshy commented on KAFKA-2046: --- Hey Clark - Onur and I are looking into this.

[jira] [Updated] (KAFKA-2046) Delete topic still doesn't work

2015-03-24 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-2046: -- Assignee: Onur Karaman (was: Sriharsha Chintalapani) Delete topic still doesn't work

Jenkins build is back to normal : Kafka-trunk #430

2015-03-24 Thread Apache Jenkins Server
See https://builds.apache.org/job/Kafka-trunk/430/changes

[jira] [Commented] (KAFKA-2046) Delete topic still doesn't work

2015-03-24 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378741#comment-14378741 ] Sriharsha Chintalapani commented on KAFKA-2046: --- [~clarkhaskins] can you add

[jira] [Assigned] (KAFKA-2046) Delete topic still doesn't work

2015-03-24 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani reassigned KAFKA-2046: - Assignee: Sriharsha Chintalapani Delete topic still doesn't work

Re: KAFKA-2042

2015-03-24 Thread Jiangjie Qin
Hi Jun, This issue does not only affect flush(). It is just with flush() the probability is much higher. It will affect the following scenario: 1. Producer started and refreshed metadata. 2. User call producer.send() to send message 1 to topic A, but message A is in accumulator. 3. User call

[jira] [Commented] (KAFKA-2032) ConsumerConfig doesn't validate partition.assignment.strategy values

2015-03-24 Thread Parth Brahmbhatt (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378834#comment-14378834 ] Parth Brahmbhatt commented on KAFKA-2032: - Created reviewboard

Review Request 32460: Patch for KAFKA-2032

2015-03-24 Thread Parth Brahmbhatt
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32460/ --- Review request for kafka. Bugs: KAFKA-2032

[jira] [Updated] (KAFKA-2032) ConsumerConfig doesn't validate partition.assignment.strategy values

2015-03-24 Thread Parth Brahmbhatt (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-2032: Attachment: KAFKA-2032.patch ConsumerConfig doesn't validate partition.assignment.strategy

Re: Review Request 31366: Patch for KAFKA-1461

2015-03-24 Thread Sriharsha Chintalapani
On March 24, 2015, 10:46 p.m., Guozhang Wang wrote: core/src/main/scala/kafka/server/AbstractFetcherThread.scala, lines 81-86 https://reviews.apache.org/r/31366/diff/2/?file=898415#file898415line81 Jun has a comment about the case when all partitions gets inactive, which is

[jira] [Commented] (KAFKA-2046) Delete topic still doesn't work

2015-03-24 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378783#comment-14378783 ] Neha Narkhede commented on KAFKA-2046: -- [~clarkhaskins] As discussed previously, the

[jira] [Created] (KAFKA-2047) Accelarate consumer rebalance in mirror maker.

2015-03-24 Thread Jiangjie Qin (JIRA)
Jiangjie Qin created KAFKA-2047: --- Summary: Accelarate consumer rebalance in mirror maker. Key: KAFKA-2047 URL: https://issues.apache.org/jira/browse/KAFKA-2047 Project: Kafka Issue Type:

Re: KAFKA-2042

2015-03-24 Thread Guozhang Wang
Ah that is right, we just need to make sure this ticket goes along with flush() call in the next release then. On Tue, Mar 24, 2015 at 3:09 PM, Jun Rao j...@confluent.io wrote: Hi, Guozhang, The flush() was added to the new producer in trunk, not in 0.8.2, right? Thanks, Jun On Tue, Mar

[jira] [Commented] (KAFKA-2042) New producer metadata update always get all topics.

2015-03-24 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378954#comment-14378954 ] Jun Rao commented on KAFKA-2042: Could you explain a bit more when the new producer will

[CANCEL] [VOTE] KIP-15 add a close method with timeout to KafkaProducer

2015-03-24 Thread Jiangjie Qin
On 3/24/15, 1:34 PM, Guozhang Wang wangg...@gmail.com wrote: +1, let's just start a new thread for this. On Tue, Mar 24, 2015 at 12:23 PM, Joel Koshy jjkosh...@gmail.com wrote: Actually, since there are already votes on this and the KIP has changed a bit we should cancel this and start a

[jira] [Created] (KAFKA-2046) Delete topic still doesn't work

2015-03-24 Thread Clark Haskins (JIRA)
Clark Haskins created KAFKA-2046: Summary: Delete topic still doesn't work Key: KAFKA-2046 URL: https://issues.apache.org/jira/browse/KAFKA-2046 Project: Kafka Issue Type: Bug

[jira] [Commented] (KAFKA-1982) change kafka.examples.Producer to use the new java producer

2015-03-24 Thread Pete Wright (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378734#comment-14378734 ] Pete Wright commented on KAFKA-1982: +1 on committing this as it will quicken adoption

Re: KAFKA-2042

2015-03-24 Thread Jun Rao
Hi, Guozhang, The flush() was added to the new producer in trunk, not in 0.8.2, right? Thanks, Jun On Tue, Mar 24, 2015 at 2:42 PM, Guozhang Wang wangg...@gmail.com wrote: Hello, We found a serious bug while testing flush() calls in the new producer, which is summarized in KAFKA-2042.

[jira] [Commented] (KAFKA-2046) Delete topic still doesn't work

2015-03-24 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378944#comment-14378944 ] Onur Karaman commented on KAFKA-2046: - I just tried deleting a topic with 128

[jira] [Created] (KAFKA-2043) CompressionType is passed in each RecordAccumulator append

2015-03-24 Thread Grant Henke (JIRA)
Grant Henke created KAFKA-2043: -- Summary: CompressionType is passed in each RecordAccumulator append Key: KAFKA-2043 URL: https://issues.apache.org/jira/browse/KAFKA-2043 Project: Kafka Issue

Review Request 32440: Patch for KAFKA-2043

2015-03-24 Thread Grant Henke
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32440/ --- Review request for kafka. Bugs: KAFKA-2043

[jira] [Commented] (KAFKA-2043) CompressionType is passed in each RecordAccumulator append

2015-03-24 Thread Grant Henke (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378046#comment-14378046 ] Grant Henke commented on KAFKA-2043: Created reviewboard

[jira] [Updated] (KAFKA-2043) CompressionType is passed in each RecordAccumulator append

2015-03-24 Thread Grant Henke (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-2043: --- Status: Patch Available (was: Open) CompressionType is passed in each RecordAccumulator append

[jira] [Updated] (KAFKA-2043) CompressionType is passed in each RecordAccumulator append

2015-03-24 Thread Grant Henke (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-2043: --- Attachment: KAFKA-2043.patch CompressionType is passed in each RecordAccumulator append

Re: New Producer Questions/Feedback

2015-03-24 Thread Grant Henke
Here is the jira: https://issues.apache.org/jira/browse/KAFKA-2043 Thanks, Grant On Mon, Mar 23, 2015 at 11:53 PM, Jun Rao j...@confluent.io wrote: RecordAccumulator is actually not part of the public api since it's internal. The public apis are only those in

[jira] [Commented] (KAFKA-1688) Add authorization interface and naive implementation

2015-03-24 Thread Parth Brahmbhatt (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378064#comment-14378064 ] Parth Brahmbhatt commented on KAFKA-1688: - [~prasadm] Thanks for taking the time

[jira] [Updated] (KAFKA-2048) java.lang.IllegalMonitorStateException thrown in AbstractFetcherThread when handling error returned from simpleConsumer

2015-03-24 Thread TAO XIAO (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TAO XIAO updated KAFKA-2048: Attachment: KAFKA-2048.patch java.lang.IllegalMonitorStateException thrown in AbstractFetcherThread when

[jira] [Created] (KAFKA-2049) Add thread that detects JVM pauses

2015-03-24 Thread Gwen Shapira (JIRA)
Gwen Shapira created KAFKA-2049: --- Summary: Add thread that detects JVM pauses Key: KAFKA-2049 URL: https://issues.apache.org/jira/browse/KAFKA-2049 Project: Kafka Issue Type: Improvement

[jira] [Updated] (KAFKA-2048) java.lang.IllegalMonitorStateException thrown in AbstractFetcherThread when handling error returned from simpleConsumer

2015-03-24 Thread TAO XIAO (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TAO XIAO updated KAFKA-2048: Status: Patch Available (was: Open) java.lang.IllegalMonitorStateException thrown in

[jira] [Updated] (KAFKA-2048) java.lang.IllegalMonitorStateException thrown in AbstractFetcherThread when handling error returned from simpleConsumer

2015-03-24 Thread TAO XIAO (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TAO XIAO updated KAFKA-2048: Attachment: KAFKA-2048.patch java.lang.IllegalMonitorStateException thrown in AbstractFetcherThread when

[jira] [Commented] (KAFKA-1293) Mirror maker housecleaning

2015-03-24 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379033#comment-14379033 ] Jiangjie Qin commented on KAFKA-1293: - [~mwarhaftig] It looks very closely related to

[jira] [Updated] (KAFKA-2048) java.lang.IllegalMonitorStateException thrown in AbstractFetcherThread when handling error returned from simpleConsumer

2015-03-24 Thread TAO XIAO (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TAO XIAO updated KAFKA-2048: Status: Patch Available (was: Open) java.lang.IllegalMonitorStateException thrown in

[jira] [Commented] (KAFKA-2029) Improving controlled shutdown for rolling updates

2015-03-24 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379175#comment-14379175 ] Jiangjie Qin commented on KAFKA-2029: - I see. Yes, there is value to put per event

[jira] [Updated] (KAFKA-2048) java.lang.IllegalMonitorStateException thrown in AbstractFetcherThread when handling error returned from simpleConsumer

2015-03-24 Thread TAO XIAO (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TAO XIAO updated KAFKA-2048: Attachment: (was: KAFKA-2048.patch) java.lang.IllegalMonitorStateException thrown in

[jira] [Updated] (KAFKA-2048) java.lang.IllegalMonitorStateException thrown in AbstractFetcherThread when handling error returned from simpleConsumer

2015-03-24 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2048: - Resolution: Fixed Status: Resolved (was: Patch Available)

[jira] [Commented] (KAFKA-2048) java.lang.IllegalMonitorStateException thrown in AbstractFetcherThread when handling error returned from simpleConsumer

2015-03-24 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379250#comment-14379250 ] Guozhang Wang commented on KAFKA-2048: -- Thanks for the patch [~xiaotao183], +1 and

[jira] [Updated] (KAFKA-2042) New producer metadata update always get all topics.

2015-03-24 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2042: - Resolution: Fixed Status: Resolved (was: Patch Available) New producer metadata update

[jira] [Commented] (KAFKA-2042) New producer metadata update always get all topics.

2015-03-24 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378986#comment-14378986 ] Jiangjie Qin commented on KAFKA-2042: - Yes, it depends on whether the topic list is

[jira] [Commented] (KAFKA-1461) Replica fetcher thread does not implement any back-off behavior

2015-03-24 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379162#comment-14379162 ] Sriharsha Chintalapani commented on KAFKA-1461: --- [~guozhang] Thanks for the

[DISCUSS] KIP-17 - Add HighwaterMarkOffset to OffsetFetchResponse

2015-03-24 Thread Grant Henke
Here is an initial proposal to add HighwaterMarkOffset to the OffsetFetchResponse: https://cwiki.apache.org/confluence/display/KAFKA/KIP-17+-+Add+HighwaterMarkOffset+to+OffsetFetchResponse I can add a jira and more implementation details if the initial proposal has interest. Thanks, Grant --

[jira] [Commented] (KAFKA-2042) New producer metadata update always get all topics.

2015-03-24 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379015#comment-14379015 ] Jun Rao commented on KAFKA-2042: If I start a console-producer w/o typing in any message,

[jira] [Commented] (KAFKA-2042) New producer metadata update always get all topics.

2015-03-24 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379120#comment-14379120 ] Jiangjie Qin commented on KAFKA-2042: - Ah, yes, that's right. I found this issue when

[jira] [Updated] (KAFKA-2048) java.lang.IllegalMonitorStateException thrown in AbstractFetcherThread when handling error returned from simpleConsumer

2015-03-24 Thread TAO XIAO (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TAO XIAO updated KAFKA-2048: Status: Open (was: Patch Available) java.lang.IllegalMonitorStateException thrown in

[jira] [Updated] (KAFKA-2048) java.lang.IllegalMonitorStateException thrown in AbstractFetcherThread when handling error returned from simpleConsumer

2015-03-24 Thread TAO XIAO (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TAO XIAO updated KAFKA-2048: Attachment: (was: KAFKA-2048.patch) java.lang.IllegalMonitorStateException thrown in

Build failed in Jenkins: KafkaPreCommit #43

2015-03-24 Thread Apache Jenkins Server
See https://builds.apache.org/job/KafkaPreCommit/43/changes Changes: [wangguoz] KAFKA-2048; Change lock synchronized to inLock() for partitionMapCond; reviewed by Guozhang Wang -- [...truncated 1194 lines...] kafka.admin.DeleteTopicTest

Re: MirrorMaker improvements

2015-03-24 Thread Guozhang Wang
Thanks for sharing this Vlad, this is great read! I am particularly interested about the last bullet point of one-to-one mapping in MM since you also mentioned that you use Kafka MM as the async replication layer for your geo-replicated k-v store. One approach that we are pursuing here to support

[jira] [Commented] (KAFKA-1856) Add PreCommit Patch Testing

2015-03-24 Thread Joe Stein (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379331#comment-14379331 ] Joe Stein commented on KAFKA-1856: -- Testing file

[jira] [Assigned] (KAFKA-1973) Remove the accidentally created LogCleanerManager.scala.orig

2015-03-24 Thread Grant Henke (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke reassigned KAFKA-1973: -- Assignee: Grant Henke Remove the accidentally created LogCleanerManager.scala.orig

[jira] [Created] (KAFKA-2044) Support requests and responses from o.a.k.common in KafkaApis

2015-03-24 Thread Gwen Shapira (JIRA)
Gwen Shapira created KAFKA-2044: --- Summary: Support requests and responses from o.a.k.common in KafkaApis Key: KAFKA-2044 URL: https://issues.apache.org/jira/browse/KAFKA-2044 Project: Kafka

[jira] [Updated] (KAFKA-1927) Replace requests in kafka.api with requests in org.apache.kafka.common.requests

2015-03-24 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1927: Attachment: KAFKA-1927.patch Replace requests in kafka.api with requests in

Review Request 32448: Patch for KAFKA-1927

2015-03-24 Thread Gwen Shapira
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32448/ --- Review request for kafka. Bugs: KAFKA-1927

[jira] [Commented] (KAFKA-1927) Replace requests in kafka.api with requests in org.apache.kafka.common.requests

2015-03-24 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378231#comment-14378231 ] Gwen Shapira commented on KAFKA-1927: - Created reviewboard

[jira] [Commented] (KAFKA-1684) Implement TLS/SSL authentication

2015-03-24 Thread Sriharsha Chintalapani (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378335#comment-14378335 ] Sriharsha Chintalapani commented on KAFKA-1684: --- [~gwenshap] if you have

[GitHub] kafka pull request: 0.8.1 migrator

2015-03-24 Thread syyang
GitHub user syyang opened a pull request: https://github.com/apache/kafka/pull/52 0.8.1 migrator You can merge this pull request into a Git repository by running: $ git pull https://github.com/uber/kafka 0.8.1-migrator Alternatively you can review and apply these changes as

[GitHub] kafka pull request: 0.8.1 migrator

2015-03-24 Thread syyang
Github user syyang closed the pull request at: https://github.com/apache/kafka/pull/52 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-1688) Add authorization interface and naive implementation

2015-03-24 Thread Don Bosco Durai (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378100#comment-14378100 ] Don Bosco Durai commented on KAFKA-1688: [~prasadm], I agree with you, we should

Re: Review Request 32440: Patch for KAFKA-2043

2015-03-24 Thread Mayuresh Gharat
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32440/#review77593 ---

[jira] [Updated] (KAFKA-2042) New producer metadata update always get all topics.

2015-03-24 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-2042: Priority: Blocker (was: Major) New producer metadata update always get all topics.

[jira] [Assigned] (KAFKA-1991) Clean ControllerStats initialization

2015-03-24 Thread Grant Henke (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke reassigned KAFKA-1991: -- Assignee: Grant Henke Clean ControllerStats initialization

Re: Review Request 32440: Patch for KAFKA-2043

2015-03-24 Thread Mayuresh Gharat
On March 24, 2015, 5 p.m., Mayuresh Gharat wrote: clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java, line 134 https://reviews.apache.org/r/32440/diff/1/?file=904417#file904417line134 Since its a Producer level config, is this change needed. We can keep

[jira] [Commented] (KAFKA-1461) Replica fetcher thread does not implement any back-off behavior

2015-03-24 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378397#comment-14378397 ] Guozhang Wang commented on KAFKA-1461: -- Sorry for the delay, I will take a look at

[jira] [Comment Edited] (KAFKA-2029) Improving controlled shutdown for rolling updates

2015-03-24 Thread Dmitry Bugaychenko (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378433#comment-14378433 ] Dmitry Bugaychenko edited comment on KAFKA-2029 at 3/24/15 7:34 PM:

[jira] [Commented] (KAFKA-2029) Improving controlled shutdown for rolling updates

2015-03-24 Thread Dmitry Bugaychenko (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378433#comment-14378433 ] Dmitry Bugaychenko commented on KAFKA-2029: --- We tried prioritization of

Re: [VOTE] KIP-15 add a close method with timeout to KafkaProducer

2015-03-24 Thread Jiangjie Qin
Push up the thread for voting after discussion on the KIP hangout. On 3/19/15, 9:03 PM, Jiangjie Qin j...@linkedin.com wrote: We had some additional discussions on the discussion thread. Pushing up this thread to resume voting. On 3/11/15, 8:47 PM, Jay Kreps jay.kr...@gmail.com wrote: Yeah

Fwd: MirrorMaker improvements

2015-03-24 Thread vlad...@gmail.com
Dear all, I had a short discussion with Jay yesterday at the ACM meetup and he suggested writing an email regarding a few possible MirrorMaker improvements. At Turn, we have been using MirrorMaker for a a few months now to asynchronously replicate our key/value store data between our

Re: KIP discussion Mar 24 at 11am PST

2015-03-24 Thread Jun Rao
Just to keep everyone posted. The following is a summary of what's being discussed in the KIP hangout today. KIP-4 (admin commands): * Gwen is uploading a patch in KAFKA-1927 (refactoring requests) so that we can get unblocked of adding new requests. * We will combine DescribeTopic and

[jira] [Comment Edited] (KAFKA-2029) Improving controlled shutdown for rolling updates

2015-03-24 Thread Dmitry Bugaychenko (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378433#comment-14378433 ] Dmitry Bugaychenko edited comment on KAFKA-2029 at 3/24/15 7:35 PM:

MirrorMaker improvements

2015-03-24 Thread vlad...@gmail.com
Dear all, I had a short discussion with Jay yesterday at the ACM meetup and he suggested writing an email regarding a few possible MirrorMaker improvements. At Turn, we have been using MirrorMaker for a a few months now to asynchronously replicate our key/value store data between our

[jira] [Created] (KAFKA-2045) Memory Management on the consumer

2015-03-24 Thread Guozhang Wang (JIRA)
Guozhang Wang created KAFKA-2045: Summary: Memory Management on the consumer Key: KAFKA-2045 URL: https://issues.apache.org/jira/browse/KAFKA-2045 Project: Kafka Issue Type: Bug

Re: [VOTE] KIP-15 add a close method with timeout to KafkaProducer

2015-03-24 Thread Joel Koshy
Actually, since there are already votes on this and the KIP has changed a bit we should cancel this and start a new thread. On Tue, Mar 24, 2015 at 12:19 PM, Jiangjie Qin j...@linkedin.com.invalid wrote: Push up the thread for voting after discussion on the KIP hangout. On 3/19/15, 9:03 PM,

Re: C++ Client Library -- libkafka-asio

2015-03-24 Thread svante karlsson
@Ewen added license.txt (boost v1.0) thanks svante 2015-03-24 2:15 GMT+01:00 Ewen Cheslack-Postava e...@confluent.io: You don't get edit permission by default, you need to get one of the admins to give you permission. @Daniel, I've added libkafka-asio. @svante I started to add

Re: Review Request 32434: Patch for KAFKA-2042

2015-03-24 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32434/#review77634 --- As discussed offline, I think it is better to make the fix in the

Re: [VOTE] KIP-15 add a close method with timeout to KafkaProducer

2015-03-24 Thread Guozhang Wang
+1, let's just start a new thread for this. On Tue, Mar 24, 2015 at 12:23 PM, Joel Koshy jjkosh...@gmail.com wrote: Actually, since there are already votes on this and the KIP has changed a bit we should cancel this and start a new thread. On Tue, Mar 24, 2015 at 12:19 PM, Jiangjie Qin

[jira] [Updated] (KAFKA-2042) New producer metadata update always get all topics.

2015-03-24 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-2042: Attachment: KAFKA-2042_2015-03-24_13:37:49.patch New producer metadata update always get all

Re: Review Request 32434: Patch for KAFKA-2042

2015-03-24 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32434/ --- (Updated March 24, 2015, 8:37 p.m.) Review request for kafka. Bugs:

[jira] [Commented] (KAFKA-2042) New producer metadata update always get all topics.

2015-03-24 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378551#comment-14378551 ] Jiangjie Qin commented on KAFKA-2042: - Updated reviewboard

[jira] [Commented] (KAFKA-2042) New producer metadata update always get all topics.

2015-03-24 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378586#comment-14378586 ] Jiangjie Qin commented on KAFKA-2042: - Updated reviewboard

Re: Review Request 32434: Patch for KAFKA-2042

2015-03-24 Thread Jiangjie Qin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32434/ --- (Updated March 24, 2015, 8:57 p.m.) Review request for kafka. Bugs:

[jira] [Commented] (KAFKA-2045) Memory Management on the consumer

2015-03-24 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378476#comment-14378476 ] Jay Kreps commented on KAFKA-2045: -- There are really two issues: 1. Bounding fetch size

[jira] [Updated] (KAFKA-1927) Replace requests in kafka.api with requests in org.apache.kafka.common.requests

2015-03-24 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-1927: Status: Patch Available (was: Open) Replace requests in kafka.api with requests in

[jira] [Commented] (KAFKA-1927) Replace requests in kafka.api with requests in org.apache.kafka.common.requests

2015-03-24 Thread Gwen Shapira (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378236#comment-14378236 ] Gwen Shapira commented on KAFKA-1927: - The uploaded patch is: 1. Preliminary and

Re: [Discussion] Using Client Requests and Responses in Server

2015-03-24 Thread Gwen Shapira
Hi, I uploaded a (very) preliminary patch with my idea. One thing thats missing: RequestResponse had handleError method that all requests implemented, typically generating appropriate error Response for the request and sending it along. Its used by KafkaApis to handle all protocol errors for

Re: Review Request 32440: Patch for KAFKA-2043

2015-03-24 Thread Grant Henke
On March 24, 2015, 5 p.m., Mayuresh Gharat wrote: clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java, line 134 https://reviews.apache.org/r/32440/diff/1/?file=904417#file904417line134 Since its a Producer level config, is this change needed. We can keep

  1   2   >