Re: Review Request 31306: Patch for KAFKA-1755

2015-02-23 Thread Joel Koshy
://reviews.apache.org/r/31306/#comment120029 In doing !compactedTopic here I'm forcing iteration over the messages below. I can also do an in-place verification here to avoid iteration (and creation of message objects). - Joel Koshy On Feb. 23, 2015, 2:43 p.m., Joel Koshy wrote

Re: Review Request 31306: Patch for KAFKA-1755

2015-02-23 Thread Joel Koshy
On Feb. 23, 2015, 7:05 p.m., Joel Koshy wrote: core/src/main/scala/kafka/message/ByteBufferMessageSet.scala, line 209 https://reviews.apache.org/r/31306/diff/1/?file=872917#file872917line209 In doing !compactedTopic here I'm forcing iteration over the messages below. I can also

Re: Review Request 31306: Patch for KAFKA-1755

2015-02-23 Thread Joel Koshy
Koshy On Feb. 23, 2015, 10:29 p.m., Joel Koshy wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31306/ --- (Updated Feb. 23, 2015

[jira] [Updated] (KAFKA-1755) Improve error handling in log cleaner

2015-02-23 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1755: -- Attachment: KAFKA-1755_2015-02-23_14:29:54.patch Improve error handling in log cleaner

[jira] [Commented] (KAFKA-1755) Improve error handling in log cleaner

2015-02-23 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14334000#comment-14334000 ] Joel Koshy commented on KAFKA-1755: --- Updated reviewboard https://reviews.apache.org/r

Re: Review Request 31306: Patch for KAFKA-1755

2015-02-23 Thread Joel Koshy
73a26377eb63ab9989698e0491049434f032cba2 Diff: https://reviews.apache.org/r/31306/diff/ Testing --- Thanks, Joel Koshy

[jira] [Commented] (KAFKA-1379) Partition reassignment resets clock for time-based retention

2015-02-23 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14334253#comment-14334253 ] Joel Koshy commented on KAFKA-1379: --- We have been thinking through various alternatives

Re: Review Request 31199: Patch for KAFKA-1965

2015-02-19 Thread Joel Koshy
/DelayedItem.scala https://reviews.apache.org/r/31199/#comment119405 May be better to name the argument delayMs - Joel Koshy On Feb. 19, 2015, 5:51 p.m., Yasuhiro Matsuda wrote: --- This is an automatically generated e-mail. To reply, visit

[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2

2015-02-19 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328493#comment-14328493 ] Joel Koshy commented on KAFKA-1729: --- Need to also update the protocol guide wiki

[jira] [Commented] (KAFKA-1546) Automate replica lag tuning

2015-02-19 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328467#comment-14328467 ] Joel Koshy commented on KAFKA-1546: --- No, we don't have any timestamp currently

Re: Review Request 30570: Patch for KAFKA-1914

2015-02-18 Thread Joel Koshy
: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30570/ --- (Updated Feb. 17, 2015, 11:46 p.m.) Review request for kafka and Joel Koshy. Bugs: KAFKA-1914 https://issues.apache.org/jira/browse

Re: [DISCUSS] KIP-8 Add a flush method to the new Java producer

2015-02-18 Thread Joel Koshy
with existing API and if you guys think this is fine then, I am ok with this approach. It is just that caller will have to do bit more work. Thanks, Bhavesh On Thursday, February 12, 2015, Joel Koshy jjkosh...@gmail.com wrote: Yes that is a counter

Re: Review Request 31169: Patch for KAFKA-1729

2015-02-18 Thread Joel Koshy
://reviews.apache.org/r/31169/#comment119238 ack - yes. thanks for catching that. - Joel Koshy On Feb. 18, 2015, 11:55 p.m., Joel Koshy wrote: --- This is an automatically generated e-mail. To reply, visit: https

[jira] [Updated] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2

2015-02-18 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1729: -- Attachment: KAFKA-1729.patch add doc for Kafka-based offset management in 0.8.2

Review Request 31169: Patch for KAFKA-1729

2015-02-18 Thread Joel Koshy
://reviews.apache.org/r/31169/diff/ Testing --- Thanks, Joel Koshy

Re: Review Request 31169: Patch for KAFKA-1729

2015-02-18 Thread Joel Koshy
to you. - Joel --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31169/#review73057 --- On Feb. 19, 2015, 1:30 a.m., Joel Koshy wrote

Review Request 31174: Patch for KAFKA-1729

2015-02-18 Thread Joel Koshy
://reviews.apache.org/r/31174/diff/ Testing --- Thanks, Joel Koshy

[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2

2015-02-18 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14326764#comment-14326764 ] Joel Koshy commented on KAFKA-1729: --- Created reviewboard https://reviews.apache.org/r

Re: Review Request 31168: Patch for KAFKA-1914

2015-02-18 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31168/#review73035 --- Ship it! Ship It! - Joel Koshy On Feb. 19, 2015, 12:01 a.m

[jira] [Updated] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2

2015-02-18 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1729: -- Attachment: KAFKA-1729_2015-02-18_17:30:37.patch add doc for Kafka-based offset management in 0.8.2

Re: Review Request 31169: Patch for KAFKA-1729

2015-02-18 Thread Joel Koshy
1c25aa3332f9e4f0222db715b524d9179b5306cf Diff: https://reviews.apache.org/r/31169/diff/ Testing --- Thanks, Joel Koshy

Re: Review Request 30848: Patch for KAFKA-1943

2015-02-17 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30848/#review72837 --- Ship it! Ship It! - Joel Koshy On Feb. 10, 2015, 10:17 p.m

Re: Review Request 31140: Patch for KAFKA-1953

2015-02-17 Thread Joel Koshy
., each expired key counts toward the aggregate even if it is all from one single producer request. - Joel Koshy On Feb. 18, 2015, 12:48 a.m., Joel Koshy wrote: --- This is an automatically generated e-mail. To reply, visit: https

Re: Review Request 31140: Patch for KAFKA-1953

2015-02-17 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31140/#review72875 --- On Feb. 18, 2015, 12:48 a.m., Joel Koshy wrote

[jira] [Assigned] (KAFKA-1962) Restore delayed request metrics

2015-02-17 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy reassigned KAFKA-1962: - Assignee: Joel Koshy I'll combine this with KAFKA-1953 Restore delayed request metrics

[jira] [Resolved] (KAFKA-1914) Count TotalProduceRequestRate and TotalFetchRequestRate in BrokerTopicMetrics

2015-02-17 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy resolved KAFKA-1914. --- Resolution: Fixed Committed to trunk Count TotalProduceRequestRate and TotalFetchRequestRate

Re: Review Request 30570: Patch for KAFKA-1914

2015-02-17 Thread Joel Koshy
/KafkaRequestHandler.scala https://reviews.apache.org/r/30570/#comment118952 I think the aggregate rates here are redundant to what's already there in RequestChannel's request metrics; but I think it is convenient to have it here as well. - Joel Koshy On Feb. 17, 2015, 11:46 p.m., Aditya Auradkar wrote

[jira] [Commented] (KAFKA-1953) Disambiguate metrics from different purgatories

2015-02-17 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14325327#comment-14325327 ] Joel Koshy commented on KAFKA-1953: --- Updated reviewboard https://reviews.apache.org/r

[jira] [Updated] (KAFKA-1953) Disambiguate metrics from different purgatories

2015-02-17 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1953: -- Attachment: KAFKA-1953_2015-02-17_18:23:55.patch Disambiguate metrics from different purgatories

Review Request 31140: Patch for KAFKA-1953

2015-02-17 Thread Joel Koshy
/main/scala/kafka/server/ReplicaManager.scala ce36cc72606fb5441335f1c7466a7db8da3db499 core/src/test/scala/unit/kafka/server/DelayedOperationTest.scala 93f52d3222fc10b6d22ef6278365f6b026180418 Diff: https://reviews.apache.org/r/31140/diff/ Testing --- Thanks, Joel Koshy

Re: Review Request 31140: Patch for KAFKA-1953

2015-02-17 Thread Joel Koshy
/ Testing --- Thanks, Joel Koshy

[jira] [Updated] (KAFKA-1953) Disambiguate metrics from different purgatories

2015-02-17 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1953: -- Status: Patch Available (was: Open) Disambiguate metrics from different purgatories

[jira] [Updated] (KAFKA-1953) Disambiguate metrics from different purgatories

2015-02-17 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1953: -- Attachment: KAFKA-1953.patch Disambiguate metrics from different purgatories

[jira] [Commented] (KAFKA-1953) Disambiguate metrics from different purgatories

2015-02-17 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14325219#comment-14325219 ] Joel Koshy commented on KAFKA-1953: --- Created reviewboard https://reviews.apache.org/r

[jira] [Resolved] (KAFKA-1960) .gitignore does not exclude test generated files and folders.

2015-02-17 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy resolved KAFKA-1960. --- Resolution: Fixed Assignee: Tong Li Thanks for the patch - committed to trunk. .gitignore

[jira] [Resolved] (KAFKA-1959) Class CommitThread overwrite group of Thread class causing compile errors

2015-02-17 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy resolved KAFKA-1959. --- Resolution: Fixed Assignee: Tong Li Thanks for the patch - committed to trunk. Class

Re: Review Request 31097: Patch for KAFKA-1960

2015-02-17 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31097/#review72911 --- Ship it! Ship It! - Joel Koshy On Feb. 16, 2015, 9:48 p.m

Re: Review Request 31088: Patch for KAFKA-1959

2015-02-17 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31088/#review72910 --- Ship it! Ship It! - Joel Koshy On Feb. 16, 2015, 4:37 p.m

Re: Review Request 27391: Fix KAFKA-1634

2015-02-17 Thread Joel Koshy
On Feb. 4, 2015, 2:15 a.m., Joel Koshy wrote: core/src/main/scala/kafka/api/OffsetCommitRequest.scala, line 48 https://reviews.apache.org/r/27391/diff/11/?file=832423#file832423line48 I our convention is to include the if in the previous line. Guozhang Wang wrote: I checked

Re: Review Request 29912: Patch for KAFKA-1852

2015-02-17 Thread Joel Koshy
On Feb. 13, 2015, 7:01 p.m., Joel Koshy wrote: core/src/main/scala/kafka/server/OffsetManager.scala, line 215 https://reviews.apache.org/r/29912/diff/3/?file=862699#file862699line215 Minor comment. I think this may be better to pass in to the OffsetManager. We should

[jira] [Created] (KAFKA-1962) Restore delayed request metrics

2015-02-17 Thread Joel Koshy (JIRA)
Joel Koshy created KAFKA-1962: - Summary: Restore delayed request metrics Key: KAFKA-1962 URL: https://issues.apache.org/jira/browse/KAFKA-1962 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-1963) Add unit tests to check presence of all metrics

2015-02-17 Thread Joel Koshy (JIRA)
Joel Koshy created KAFKA-1963: - Summary: Add unit tests to check presence of all metrics Key: KAFKA-1963 URL: https://issues.apache.org/jira/browse/KAFKA-1963 Project: Kafka Issue Type: Bug

[jira] [Commented] (KAFKA-1961) Looks like its possible to delete _consumer_offsets topic

2015-02-17 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14324917#comment-14324917 ] Joel Koshy commented on KAFKA-1961: --- Yes it would be inconsistent in that you would lose

Re: Review Request 29912: Patch for KAFKA-1852

2015-02-13 Thread Joel Koshy
out offsets on a delete topic is done yet - Onur Karaman did it for ZK based offsets but we need a separate jira to delete Kafka-based offsets. - Joel Koshy On Feb. 13, 2015, 12:46 a.m., Sriharsha Chintalapani wrote

[jira] [Updated] (KAFKA-1697) remove code related to ack1 on the broker

2015-02-13 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1697: -- Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the patch. Pushed

[jira] [Created] (KAFKA-1953) Disambiguate metrics from different purgatories

2015-02-13 Thread Joel Koshy (JIRA)
Joel Koshy created KAFKA-1953: - Summary: Disambiguate metrics from different purgatories Key: KAFKA-1953 URL: https://issues.apache.org/jira/browse/KAFKA-1953 Project: Kafka Issue Type: Sub-task

Re: [DISCUSS] KIP-8 Add a flush method to the new Java producer

2015-02-12 Thread Joel Koshy
/api/java/io/OutputStream.html#flush() -Jay On Tue, Feb 10, 2015 at 10:41 AM, Joel Koshy jjkosh...@gmail.com wrote: I think tryFlush with a timeout sounds good to me. This is really more for consistency than anything else. I cannot think of any standard blocking calls off the top of my

Re: [DISCUSSION] KIP-2: Refactor Brokers to Allow Multiple Endpoints

2015-02-12 Thread Joel Koshy
to use.new.wire.protocol. On Wed, Feb 11, 2015 at 4:19 PM, Joel Koshy jjkosh...@gmail.com wrote: The description that Jun gave for (2) was the detail I was looking for - Gwen can you update the KIP with that for completeness/clarity? I'm +1 as well overall

Re: Review Request 29647: Patch for KAFKA-1697

2015-02-12 Thread Joel Koshy
that up. - Joel Koshy On Feb. 13, 2015, 2:57 a.m., Gwen Shapira wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29647

Re: gradle testAll stuck

2015-02-12 Thread Joel Koshy
- Can you enable test logging (see the README) and see if you can figure out which test is getting stuck or taking forever? - A thread-dump may help. On Thu, Feb 12, 2015 at 08:57:11AM -0500, Tong Li wrote: Hi, folks, How are you all doing? New bee here. Run gradle --daemon

Re: Review Request 29647: Patch for KAFKA-1697

2015-02-12 Thread Joel Koshy
. core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala https://reviews.apache.org/r/29647/#comment118160 Is this change necessary? - Joel Koshy On Feb. 12, 2015, 7:14 a.m., Gwen Shapira wrote

Re: Review Request 29912: Patch for KAFKA-1852

2015-02-12 Thread Joel Koshy
://reviews.apache.org/r/29912/#comment118171 Can we just add an exists(topic) method to metadataCache? That way we can just do something like offsetMetadata.groupBy((topicPartition, offsetMetadata) = metadataCache.contains(topicPartition.topic)) - Joel Koshy On Jan. 19, 2015, 6:44 p.m

Re: Review Request 29647: Patch for KAFKA-1697

2015-02-12 Thread Joel Koshy
On Feb. 12, 2015, 2:31 p.m., Joel Koshy wrote: core/src/main/scala/kafka/server/ReplicaManager.scala, line 271 https://reviews.apache.org/r/29647/diff/6/?file=861625#file861625line271 This is good, but maybe call this canRespondNow ? Since before replication won't apply

Re: [KIP-DISCUSSION] Mirror Maker Enhancement

2015-02-11 Thread Joel Koshy
, 2015 at 12:31 PM, Joel Koshy jjkosh...@gmail.com wrote: On Tue, Feb 10, 2015 at 12:13:46PM -0800, Neha Narkhede wrote: I think all of us agree that we want to design MirrorMaker for 0 data loss. With the absence of the data channel, 0 data loss will be much simpler to implement

[jira] [Commented] (KAFKA-1852) OffsetCommitRequest can commit offset on unknown topic

2015-02-11 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14316739#comment-14316739 ] Joel Koshy commented on KAFKA-1852: --- Thanks for the ping - will take a look

[jira] [Commented] (KAFKA-1374) LogCleaner (compaction) does not support compressed topics

2015-02-11 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14316714#comment-14316714 ] Joel Koshy commented on KAFKA-1374: --- I can review this next week. However, as far

[jira] [Resolved] (KAFKA-1945) MetaData Response - Broker hostname is wrong

2015-02-11 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy resolved KAFKA-1945. --- Resolution: Invalid MetaData Response - Broker hostname is wrong

[jira] [Commented] (KAFKA-1944) Rename LogCleaner and related classes to LogCompactor

2015-02-11 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14316222#comment-14316222 ] Joel Koshy commented on KAFKA-1944: --- Sure - I think this can wait until

Re: [KIP-DISCUSSION] Mirror Maker Enhancement

2015-02-11 Thread Joel Koshy
. If so let's flag it while we still have leeway on the consumer. If we think that will work, well I do think it is conceptually a lot simpler than the current code, though I suppose one could disagree on that. -Jay On Wed, Feb 11, 2015 at 5:53 AM, Joel Koshy jjkosh...@gmail.com wrote: Hi

Re: [DISCUSSION] KIP-2: Refactor Brokers to Allow Multiple Endpoints

2015-02-11 Thread Joel Koshy
The description that Jun gave for (2) was the detail I was looking for - Gwen can you update the KIP with that for completeness/clarity? I'm +1 as well overall. However, I think it would be good if we also get an ack from someone who is more experienced on the operations side (say, Todd) to

Re: [DISCUSSION] KAFKA-1697 - make NotEnoughReplicasAfterAppend a non-retriable exception

2015-02-11 Thread Joel Koshy
Thanks for the comments - however, it is not clear to me what your preference is on making NotEnoughReplicasAfterAppend retriable vs non-retriable. As for me, my preference is to leave it as retriable since it is clear that the produce may succeed on a retry (and may introduce a duplicate). I

Re: Review Request 29647: Patch for KAFKA-1697

2015-02-11 Thread Joel Koshy
://reviews.apache.org/r/29647/#comment117873 Do we need this here? - Joel Koshy On Feb. 11, 2015, 1:06 a.m., Gwen Shapira wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29647

Re: Review Request 30810: Patch for KAFKA-1937

2015-02-10 Thread Joel Koshy
://reviews.apache.org/r/30810/#comment117707 previously mentioned race - if a message is held by the producer thread by not handed to the producer - Joel Koshy On Feb. 9, 2015, 11:58 p.m., Jiangjie Qin wrote: --- This is an automatically

Re: [KIP-DISCUSSION] Mirror Maker Enhancement

2015-02-10 Thread Joel Koshy
I think the message handler adds little to no complexity to the mirror maker. Jay/Neha, the MM became scary due to the rearchitecture we did for 0.8 due to performance issues compared with 0.7 - we should remove the data channel if it can match the current throughput. I agree it is worth

Re: [DISCUSS] KIP-8 Add a flush method to the new Java producer

2015-02-10 Thread Joel Koshy
of flush there is no way to say that. As you say you only may that penalty on one of the get() calls, but if the linger.ms is high (say 60 seconds) that will be a huge penalty. -Jay On Mon, Feb 9, 2015 at 6:23 PM, Joel Koshy jjkosh...@gmail.com wrote: - WRT the motivation: if you set linger.ms

Re: Review Request 30810: Patch for KAFKA-1937

2015-02-10 Thread Joel Koshy
://reviews.apache.org/r/30810/#comment117705 The node itself may not be inserted into the list yet, so the current implementation of remove could throw an NPE - Joel Koshy On Feb. 9, 2015, 11:58 p.m., Jiangjie Qin wrote

Re: [DISCUSS] KIP-8 Add a flush method to the new Java producer

2015-02-10 Thread Joel Koshy
I think tryFlush with a timeout sounds good to me. This is really more for consistency than anything else. I cannot think of any standard blocking calls off the top of my head that don't have a timed variant. E.g., Thread.join, Object.wait, Future.get Either that, or they provide an entirely

Re: Review Request 29647: Patch for KAFKA-1697

2015-02-10 Thread Joel Koshy
the appendMessages call. core/src/main/scala/kafka/server/KafkaApis.scala https://reviews.apache.org/r/29647/#comment117712 Should we just do Errors.INVALID_REQUIRED_ACKS? - Joel Koshy On Jan. 14, 2015, 11:41 p.m., Gwen Shapira wrote

Re: [KIP-DISCUSSION] Mirror Maker Enhancement

2015-02-10 Thread Joel Koshy
the KIP still refers to the data channel in a few places (Motivation and On consumer rebalance sections). Can you update the wiki so it is easier to review the new design, especially the data loss part. On Tue, Feb 10, 2015 at 10:36 AM, Joel Koshy jjkosh...@gmail.com wrote: I think the message

Re: Review Request 29647: Patch for KAFKA-1697

2015-02-10 Thread Joel Koshy
already has special cases for requiredAcks == 0 - however, that code is executed in the callback from replica manager. So it's not a 100% clean separation. - Joel Koshy On Jan. 14, 2015, 11:41 p.m., Gwen Shapira wrote

Re: Review Request 29647: Patch for KAFKA-1697

2015-02-10 Thread Joel Koshy
/NotEnoughReplicasAfterAppendException.java https://reviews.apache.org/r/29647/#comment117734 Understood, but if someone uses required.acks -1 they would most likely be okay with duplicates and would rather have the data persisted with guarantees. What do you think? - Joel Koshy On Jan. 14, 2015, 11:41

Re: Review Request 29647: Patch for KAFKA-1697

2015-02-10 Thread Joel Koshy
on NotEnoughReplicasAfterAppendException. The side-effect in question here is duplicates. Duplicates can arise even for other errors (e.g., request timed out). So that side-effect is not compelling enough to warrant a change to make this non-retriable. - Joel Koshy On Jan. 14, 2015, 11:41 p.m., Gwen Shapira wrote

Re: [DISCUSSION] KIP-2: Refactor Brokers to Allow Multiple Endpoints

2015-02-09 Thread Joel Koshy
For (1) - +1 especially since the existing clients will keep working. For (2) - I'm less clear on the proposal. Can you incorporate it into the KIP and/or linked wiki? Also, on the KIP itself, can you clarify what the TRACE protocol is? The RB has a brief comment (plan is to add instrumentation

Re: [DISCUSS] KIP-8 Add a flush method to the new Java producer

2015-02-09 Thread Joel Koshy
- WRT the motivation: if you set linger.ms 0 to encourage batching of messages, which is likely a good idea for this kind of use case, then the second for loop will block for a ms - however, in practice this will really only be for the first couple of calls right? Since the subsequent

Re: [DISCUSS] KIPs

2015-02-09 Thread Joel Koshy
-ordinate/bug people but will end up being worth it because each person on the project will have a holistic picture of what is going on. -Jay On Thu, Feb 5, 2015 at 11:24 PM, Joel Koshy jjkosh...@gmail.com wrote: Just wanted to add a few more comments on this: KIPs were suggested

Re: Review Request 30570: Patch for KAFKA-1914

2015-02-09 Thread Joel Koshy
the discrepancies. E.g., right now offset commit failures due to append failures are counted in failed produce rate. - Joel Koshy On Feb. 3, 2015, 7:13 p.m., Aditya Auradkar wrote: --- This is an automatically generated e-mail. To reply, visit

[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2

2015-02-06 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14309817#comment-14309817 ] Joel Koshy commented on KAFKA-1729: --- I will be making a few more edits after some

Re: [DISCUSS] KIPs

2015-02-05 Thread Joel Koshy
. Why not go with usual definition: lazy = No strong objections for few days? This means contributors will not be blocked on issues where no one cares to comment (and we had few of those). Gwen On Thu, Feb 5, 2015 at 4:14 PM, Joel Koshy jjkosh

Re: [DISCUSS] KIPs

2015-02-05 Thread Joel Koshy
now that 0.8.2 is out of the way. Thanks, Joel On Thu, Feb 05, 2015 at 08:56:13PM -0800, Joel Koshy wrote: I'm just thinking aloud - I don't know what a good number would be, and it is just one possibility to streamline how KIPs are processed. It largely depends on how complex the proposals

Re: [DISCUSS] KIPs

2015-02-05 Thread Joel Koshy
wrote: +1 On Thu, Feb 5, 2015 at 6:16 PM, Neha Narkhede n...@confluent.io wrote: Sounds good. On Thu, Feb 5, 2015 at 2:35 PM, Jay Kreps jay.kr...@gmail.com wrote: None on my part. -Jay On Thu, Feb 5, 2015 at 11:50 AM, Joel Koshy jjkosh...@gmail.com wrote: One

Re: [DISCUSS] KIPs

2015-02-05 Thread Joel Koshy
and understand the change before you proceed. I basically think all of us should be reading these proposals and giving feedback. -Jay On Thu, Feb 5, 2015 at 4:14 PM, Joel Koshy jjkosh...@gmail.com wrote: Sorry about this - I actually meant to suggest lazy consensus (which is a stronger

Re: [DISCUSS] KIPs

2015-02-05 Thread Joel Koshy
no one cares to comment (and we had few of those). Gwen On Thu, Feb 5, 2015 at 4:14 PM, Joel Koshy jjkosh...@gmail.com wrote: Sorry about this - I actually meant to suggest lazy consensus (which is a stronger requirement): 3 binding +1 votes and no binding vetoes. I have updated

Re: [DISCUSS] KIPs

2015-02-05 Thread Joel Koshy
, Feb 5, 2015 at 4:14 PM, Joel Koshy jjkosh...@gmail.com wrote: Sorry about this - I actually meant to suggest lazy consensus (which is a stronger requirement): 3 binding +1 votes and no binding vetoes. I have updated the wiki to lazy consensus - but can change it back

Re: [DISCUSS] KIPs

2015-02-05 Thread Joel Koshy
?) and/or the version the KIP was first released in. On Fri, Jan 16, 2015 at 9:20 AM, Joel Koshy jjkosh...@gmail.com wrote: I'm definitely +1 on the KIP concept. As Joe mentioned, we are already doing this in one form or the other. However, IMO it is fairly

Re: kafka.utils.DoublyLinkedList

2015-02-04 Thread Joel Koshy
Jay, My understanding is that java linked lists (even though they are backed by a doubly linked list) support O(1) deletion only if you have the index of the object. If you do a remove(object) it becomes O(n). As for scala lists - the scaladocs seem to indicate that the doubly linked lists are

[jira] [Created] (KAFKA-1922) Move doubly linked list test out of UtilsTest

2015-02-04 Thread Joel Koshy (JIRA)
Joel Koshy created KAFKA-1922: - Summary: Move doubly linked list test out of UtilsTest Key: KAFKA-1922 URL: https://issues.apache.org/jira/browse/KAFKA-1922 Project: Kafka Issue Type: Bug

Re: Review Request 30547: Integrate checkstyle.

2015-02-03 Thread Joel Koshy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30547/#review70911 --- Ship it! Ship It! - Joel Koshy On Feb. 3, 2015, 5:17 p.m., Jay

Re: Review Request 27391: Fix KAFKA-1634

2015-02-03 Thread Joel Koshy
. - Joel Koshy On Jan. 24, 2015, 12:06 a.m., Guozhang Wang wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27391/ --- (Updated Jan

Re: Review Request 30547: Integrate checkstyle.

2015-02-03 Thread Joel Koshy
On Feb. 3, 2015, 7:44 a.m., Guozhang Wang wrote: clients/src/main/java/org/apache/kafka/common/metrics/JmxReporter.java, line 45 https://reviews.apache.org/r/30547/diff/1/?file=845037#file845037line45 Wondering why LOCK needs to be all-capitalized while Log above does not need

[jira] [Commented] (KAFKA-1916) Kafka unit tests need to choose random port

2015-02-03 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304514#comment-14304514 ] Joel Koshy commented on KAFKA-1916: --- Dup of KAFKA-1501 ? Kafka unit tests need

[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2

2015-02-02 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14301864#comment-14301864 ] Joel Koshy commented on KAFKA-1729: --- Yes I will commit it today. Re: rolling back to ZK

Re: Review Request 28769: Patch for KAFKA-1809

2015-02-02 Thread Joel Koshy
. Seems to be an issue with the patch review script as you normally shouldn't need to git pull before submitting. - Joel Koshy On Feb. 2, 2015, 7:55 p.m., Gwen Shapira wrote: --- This is an automatically generated e-mail. To reply

[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2

2015-02-02 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14302676#comment-14302676 ] Joel Koshy commented on KAFKA-1729: --- I committed the patch to trunk - but then realized

[jira] [Commented] (KAFKA-1870) Cannot commit with simpleConsumer on Zookeeper only with Java API

2015-02-02 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14302679#comment-14302679 ] Joel Koshy commented on KAFKA-1870: --- [~junrao] (at least on trunk) I think the change

Re: Review Request 30547: Integrate checkstyle.

2015-02-02 Thread Joel Koshy
checkstyle docs but probably missed it. checkstyle/import-control.xml https://reviews.apache.org/r/30547/#comment116069 Is there a notion of a catch-all? Say if we were to add a new subpackage or package and need to add specific allow/disallow rules to it but forget to do that. - Joel Koshy

[jira] [Created] (KAFKA-1911) Log deletion on stopping replicas should be async

2015-01-31 Thread Joel Koshy (JIRA)
Joel Koshy created KAFKA-1911: - Summary: Log deletion on stopping replicas should be async Key: KAFKA-1911 URL: https://issues.apache.org/jira/browse/KAFKA-1911 Project: Kafka Issue Type: Bug

[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2

2015-01-29 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14298059#comment-14298059 ] Joel Koshy commented on KAFKA-1729: --- Thanks Jun - will do. When you get a chance, take

[jira] [Commented] (KAFKA-1901) Move Kafka version to be generated in code by build (instead of in manifest)

2015-01-28 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14295566#comment-14295566 ] Joel Koshy commented on KAFKA-1901: --- Can you clarify what you mean by metadata fetch

[jira] [Updated] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2

2015-01-28 Thread Joel Koshy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Koshy updated KAFKA-1729: -- Description: (was: [~junrao] before you roll out another 0.8.2 rc can you review this?) add doc

Re: Review Request 30321: Patch for kafka-1902

2015-01-28 Thread Joel Koshy
/KafkaMetricsGroup.scala https://reviews.apache.org/r/30321/#comment114965 since some reporters (such as graphite) I don't think jmx reporter cares about dot (or does it?) - Joel Koshy On Jan. 28, 2015, 5:23 p.m., Jun Rao wrote

<    1   2   3   4   5   6   7   8   9   10   >