Review Request 20372: Patch for KAFKA-1390

2014-04-15 Thread Jun Rao
/kafka/server/ServerShutdownTest.scala c7e058fc2e541019fcfb666d556050a13c7d4e66 core/src/test/scala/unit/kafka/utils/TestUtils.scala 53d01aa3e501392672ee6bc7f0e225493383c870 Diff: https://reviews.apache.org/r/20372/diff/ Testing --- Thanks, Jun Rao

Re: Review Request 20380: Fix KAFKA-1395

2014-04-16 Thread Jun Rao
/AutoOffsetResetTest.scala https://reviews.apache.org/r/20380/#comment73567 Do we really need this? Metadata is propagated after the leader is elected in ZK. - Jun Rao On April 15, 2014, 10:07 p.m., Guozhang Wang wrote: --- This is an automatically

Re: Review Request 20252: Patch for KAFKA-1356

2014-04-17 Thread Jun Rao
due to a few tests that have delete topic enabled. Other than that, unit tests are passing. - Jun Rao On April 11, 2014, 7:42 a.m., Timothy Chen wrote: --- This is an automatically generated e-mail. To reply, visit: https

Re: Review Request 20471: Patch for KAFKA-1398

2014-04-18 Thread Jun Rao
to achieve that. core/src/test/scala/unit/kafka/server/DynamicConfigChangeTest.scala https://reviews.apache.org/r/20471/#comment73901 Missing Apache header. - Jun Rao On April 18, 2014, 12:36 a.m., Jay Kreps wrote

Re: Review Request 20471: Patch for KAFKA-1398

2014-04-18 Thread Jun Rao
On April 18, 2014, 3:22 p.m., Jun Rao wrote: core/src/main/scala/kafka/server/TopicConfigManager.scala, lines 123-124 https://reviews.apache.org/r/20471/diff/1/?file=561891#file561891line123 This is not needed. In ZK, new sequential nodes always get a higher id whether previous

Re: Review Request 20440: Patch for KAFKA-1337

2014-04-18 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20440/#review40790 --- Ship it! Ship It! - Jun Rao On April 17, 2014, 12:41 a.m., Jay

Re: Review Request 20376: Publish source jars and javadoc

2014-04-18 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20376/#review40791 --- Ship it! Ship It! - Jun Rao On April 15, 2014, 7:39 p.m., Joel

Re: Review Request 20471: KAFKA-1398: Follow-up comments on dynamic config

2014-04-18 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20471/#review40810 --- Ship it! Ship It! - Jun Rao On April 18, 2014, 8:03 p.m., Jay

Re: Review Request 20468: Add log cleaner metrics. (Cherry-picked from trunk)

2014-04-18 Thread Jun Rao
https://reviews.apache.org/r/20468/#comment74050 It seems that we should set the ratio back to 0 if dirtyLogs is empty. Otherwise, the reported metrics will stay with the last updated value, which can be miss-leading. - Jun Rao On April 17, 2014, 10:33 p.m., Joel Koshy wrote

Re: Review Request 20468: Add log cleaner metrics. (Cherry-picked from trunk)

2014-04-18 Thread Jun Rao
. CleanerStats.startTime, mapCompleteTime, endTime, bytesRead, bytesWritten, mapBytesRead, mapMessagesRead, messagesRead, messagesWritten LogCleanerManager.dirtiestLogCleanableRatio Jay, Could you double check? - Jun Rao On April 17, 2014, 10:33 p.m., Joel Koshy wrote

Re: Review Request 20534: Patch for KAFKA-1380

2014-04-21 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20534/#review40971 --- Ship it! Ship It! - Jun Rao On April 21, 2014, 9:47 p.m., Joel

Review Request 20540: Patch for KAFKA-1410

2014-04-21 Thread Jun Rao
/scala/kafka/server/ReplicaManager.scala 5588f59783d5cd8f074263c207045625282af12d core/src/test/scala/unit/kafka/server/SimpleFetchTest.scala 17b08e14f5e9bfec1233c80d74b50bc2ec38aa25 Diff: https://reviews.apache.org/r/20540/diff/ Testing --- Thanks, Jun Rao

Review Request 20541: Patch for KAFKA-1411

2014-04-21 Thread Jun Rao
core/src/test/scala/unit/kafka/integration/PrimitiveApiTest.scala a062f68c7f6b285fb0c63cec238882c9bc7ee246 core/src/test/scala/unit/kafka/producer/AsyncProducerTest.scala 18e355501808c00b610da8346e0dd1898d5b9301 Diff: https://reviews.apache.org/r/20541/diff/ Testing --- Thanks, Jun

Re: Review Request 20538: Patch for KAFKA-1380

2014-04-21 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20538/#review40978 --- Ship it! Ship It! - Jun Rao On April 21, 2014, 10:40 p.m., Joel

Re: Review Request 20537: Patch for KAFKA-1400

2014-04-21 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20537/#review40987 --- Ship it! Ship It! - Jun Rao On April 21, 2014, 10:27 p.m., Neha

treating unit tests as code

2014-04-22 Thread Jun Rao
Recently, we have seen multiple transient unit test failures. While fixing those issues, I realized that one of the problems that led to this situation is that we haven't been treating unit tests in the same way as the rest of the code. I found myself guilty of this too. When reviewing patches, I

Re: Review Request 20540: Patch for KAFKA-1410

2014-04-22 Thread Jun Rao
5588f59783d5cd8f074263c207045625282af12d core/src/test/scala/unit/kafka/server/SimpleFetchTest.scala 17b08e14f5e9bfec1233c80d74b50bc2ec38aa25 Diff: https://reviews.apache.org/r/20540/diff/ Testing --- Thanks, Jun Rao

Re: Review Request 20541: Patch for KAFKA-1411

2014-04-22 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20541/#review40981 --- On April 21, 2014, 11:34 p.m., Jun Rao wrote

Re: Review Request 20540: Patch for KAFKA-1410

2014-04-22 Thread Jun Rao
/server/ReplicaManager.scala 5588f59783d5cd8f074263c207045625282af12d core/src/test/scala/unit/kafka/server/SimpleFetchTest.scala 17b08e14f5e9bfec1233c80d74b50bc2ec38aa25 Diff: https://reviews.apache.org/r/20540/diff/ Testing --- Thanks, Jun Rao

Re: Review Request 20540: Patch for KAFKA-1410

2014-04-22 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20540/#review41039 --- On April 22, 2014, 8:51 p.m., Jun Rao wrote

Re: Review Request 20540: Patch for KAFKA-1410

2014-04-22 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20540/#review40980 --- On April 22, 2014, 8:51 p.m., Jun Rao wrote

Re: Review Request 20380: Fix KAFKA-1395: incorporate Neha's comment, ensure createProducer use ack = -1 and type = async, with configurable serializer and partitioner

2014-04-22 Thread Jun Rao
/#comment74457 Same as the above. Sensitive to # produced messages and need to create topic first. - Jun Rao On April 22, 2014, 5:43 p.m., Guozhang Wang wrote: --- This is an automatically generated e-mail. To reply, visit: https

Re: a problem about kafka source code‏

2014-04-23 Thread Jun Rao
It seems to call unlock in the finally clause. Thanks, Jun On Wed, Apr 23, 2014 at 6:12 AM, 冯傲风 faof0...@outlook.com wrote: HiI am reading the kafka source code now. I have a problem about the BufferPool class int the file org.apache.kafka.clients.producer.internals.BufferPool.javathe

Re: Review Request 20290: Patch for KAFKA-1389

2014-04-23 Thread Jun Rao
: https://reviews.apache.org/r/20290/diff/ Testing --- Thanks, Jun Rao

Re: Review Request 20290: Patch for KAFKA-1389

2014-04-23 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20290/#review40247 --- On April 23, 2014, 5:41 p.m., Jun Rao wrote

Re: Review Request 20616: Fix KAFKA-1409

2014-04-23 Thread Jun Rao
messages. Perhaps we can collect the stat for the incoming byte/message rate after the log append. - Jun Rao On April 23, 2014, 5:29 p.m., Guozhang Wang wrote: --- This is an automatically generated e-mail. To reply, visit: https

Re: Review Request 20290: Patch for KAFKA-1389

2014-04-23 Thread Jun Rao
4bd59645635cb693de3a638e58ef19363eb742e5 Diff: https://reviews.apache.org/r/20290/diff/ Testing --- Thanks, Jun Rao

Re: Review Request 20616: Fix KAFKA-1409: Address Jun's comments

2014-04-23 Thread Jun Rao
://reviews.apache.org/r/20616/#comment74653 Could we revert the change in this file? - Jun Rao On April 23, 2014, 9:26 p.m., Guozhang Wang wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r

Re: Review Request 20290: Patch for KAFKA-1389

2014-04-23 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20290/#review41220 --- On April 23, 2014, 9:36 p.m., Jun Rao wrote: --- This is an automatically generated e-mail

Re: Review Request 20616: Fix KAFKA-1409: Address Jun's comments round two

2014-04-23 Thread Jun Rao
Could this be changed to validBytes too? core/src/main/scala/kafka/log/Log.scala https://reviews.apache.org/r/20616/#comment74689 This requires computing the checksum and can be expensive. We probably should move it to after the message size check. - Jun Rao On April 23, 2014, 10:20 p.m

Review Request 20677: Patch for KAFKA-1392

2014-04-24 Thread Jun Rao
/test/scala/unit/kafka/utils/TestUtils.scala 130b6be8414268b9a992514ecbc7561872ccaca7 Diff: https://reviews.apache.org/r/20677/diff/ Testing --- Thanks, Jun Rao

Re: Review Request 20677: Patch for KAFKA-1392

2014-04-24 Thread Jun Rao
, 2014, 6:08 p.m., Jun Rao wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20677/ --- (Updated April 24, 2014, 6:08 p.m

Review Request 20695: Patch for KAFKA-1392

2014-04-24 Thread Jun Rao
481a400067ec97ed97a2bbd8506b34960a5ec000 core/src/test/scala/unit/kafka/utils/TestUtils.scala 130b6be8414268b9a992514ecbc7561872ccaca7 Diff: https://reviews.apache.org/r/20695/diff/ Testing --- Thanks, Jun Rao

Re: Review Request 20695: Patch for KAFKA-1392

2014-04-24 Thread Jun Rao
--- Thanks, Jun Rao

Re: Review Request 20616: Fix KAFKA-1409: Address Jun's comments round three

2014-04-24 Thread Jun Rao
On April 24, 2014, 12:44 a.m., Jun Rao wrote: core/src/main/scala/kafka/log/Log.scala, lines 275-276 https://reviews.apache.org/r/20616/diff/4/?file=566280#file566280line275 We probably should use validMessages here. Guozhang Wang wrote: I actually thought about

Review Request 20713: Patch for KAFKA-1424

2014-04-25 Thread Jun Rao
, Jun Rao

Re: Review Request 20380: Fix KAFKA-1395: Address Jun's comments

2014-04-25 Thread Jun Rao
. TestUtils.createProducer: Could we keep the props parameter and treat it as the list of properties that we want to override? 2. Unused imports in ServerShutdownTest and LogRecoveryTest. - Jun Rao On April 24, 2014, 11:22 p.m., Guozhang Wang wrote

Re: [VOTE] Apache Kafka 0.8.1.1 Release Candidate 1

2014-04-25 Thread Jun Rao
+1. Verified quick start. Thanks, Jun On Tue, Apr 22, 2014 at 4:18 PM, Joe Stein joe.st...@stealth.ly wrote: This is the first candidate for release of Apache Kafka 0.8.1.1 Release Notes for the 0.8.1.1 release

Re: Review Request 20616: Fix KAFKA-1409: Address Jun and Neha's comments

2014-04-25 Thread Jun Rao
/r/20616/#comment74934 entrySNize? core/src/main/scala/kafka/log/Log.scala https://reviews.apache.org/r/20616/#comment74936 entrySNnnnNize? - Jun Rao On April 25, 2014, 5:22 p.m., Guozhang Wang wrote

Re: Review Request 20380: Fix KAFKA-1395: Address Jun's comments

2014-04-25 Thread Jun Rao
/unit/kafka/utils/TestUtils.scala https://reviews.apache.org/r/20380/#comment74967 Could we add a comment on producerProps? - Jun Rao On April 25, 2014, 6:03 p.m., Guozhang Wang wrote: --- This is an automatically generated e-mail

Re: [VOTE RESULT] Apache Kafka 0.8.1.1 Release Candidate 1

2014-04-26 Thread Jun Rao
start + unit tests. On 4/25/14 10:54 AM, Neha Narkhede neha.narkh...@gmail.com wrote: +1. Verified quick start. Thanks, Neha On Fri, Apr 25, 2014 at 8:39 AM, Jun Rao jun...@gmail.com wrote: +1. Verified quick start. Thanks, Jun On Tue, Apr 22, 2014 at 4

Re: Review Request 19731: Patch for KAFKA-1328

2014-04-27 Thread Jun Rao
/main/java/org/apache/kafka/clients/consumer/OffsetMetadata.java https://reviews.apache.org/r/19731/#comment75061 It seems it's more natural to return an error code than an exception. - Jun Rao On April 13, 2014, 2:12 a.m., Neha Narkhede wrote

Review Request 20783: Patch for KAFKA-1424

2014-04-28 Thread Jun Rao
/admin/AdminTest.scala a8d92f678bed7cb36bb29bdeeb772782cc793466 core/src/test/scala/unit/kafka/utils/TestUtils.scala f5a7a5bd92202d805250d9356c8a4e89e6f18732 Diff: https://reviews.apache.org/r/20783/diff/ Testing --- Thanks, Jun Rao

Re: Review Request 20783: Patch for KAFKA-1424

2014-04-28 Thread Jun Rao
://reviews.apache.org/r/20783/diff/ Testing --- Thanks, Jun Rao

Review Request 20787: Patch for KAFKA-1383

2014-04-28 Thread Jun Rao
- core/src/test/scala/unit/kafka/network/SocketServerTest.scala 75bd41b7d0ccbd3763e21905173f05f43f2a02f2 Diff: https://reviews.apache.org/r/20787/diff/ Testing --- Thanks, Jun Rao

Re: Review Request 20745: Patch for KAFKA-1397

2014-04-29 Thread Jun Rao
/#comment75296 Not sure if this adds any value than the previous send. - Jun Rao On April 29, 2014, 12:08 a.m., Timothy Chen wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20745

Re: Review Request 20745: Patch for KAFKA-1397

2014-04-29 Thread Jun Rao
On April 28, 2014, 7:43 p.m., Neha Narkhede wrote: core/src/main/scala/kafka/controller/ControllerChannelManager.scala, line 210 https://reviews.apache.org/r/20745/diff/1/?file=569160#file569160line210 better to not leak the logic from delete topic manager here. It is worth

Review Request 20930: Patch for KAFKA-1433

2014-04-30 Thread Jun Rao
, Jun Rao

Re: Review Request 20927: Added metric to track the minimum offset for each topic/partition

2014-05-01 Thread Jun Rao
/r/20927/#comment75570 Wouldn't it be cheaper to do logSegments.head.baseOffset? - Jun Rao On April 30, 2014, 11:26 p.m., Alex Couture-Beil wrote: --- This is an automatically generated e-mail. To reply, visit: https

Re: Review Request 20745: Patch for KAFKA-1397

2014-05-01 Thread Jun Rao
/unit/kafka/admin/DeleteTopicTest.scala https://reviews.apache.org/r/20745/#comment75702 The message should be partition reassignment shouldn't complete. - Jun Rao On May 1, 2014, 10:54 p.m., Timothy Chen wrote

Re: Review Request 20718: Patch for KAFKA-1384

2014-05-01 Thread Jun Rao
/scala/kafka/server/KafkaServerStartable.scala https://reviews.apache.org/r/20718/#comment75720 What's the intention of this method? - Jun Rao On April 26, 2014, 7:09 a.m., Timothy Chen wrote: --- This is an automatically generated

Re: Review Request 20997: Fix KAFKA-1432

2014-05-01 Thread Jun Rao
://reviews.apache.org/r/20997/#comment75734 Why are we setting producerThreads to empty? perf/src/main/scala/kafka/perf/ProducerPerformance.scala https://reviews.apache.org/r/20997/#comment75739 Is there a reason that we change to send a null key? Will that break system tests? - Jun Rao On May

Re: Review Request 20745: Patch for KAFKA-1397

2014-05-02 Thread Jun Rao
addressed. Could you make another pass? If they don't need to be addressed, could you add a comment? - Jun Rao On May 2, 2014, 1:12 a.m., Timothy Chen wrote: --- This is an automatically generated e-mail. To reply, visit: https

Re: Review Request 20745: Patch for KAFKA-1397

2014-05-05 Thread Jun Rao
://reviews.apache.org/r/20745/#comment75798 You can revert the changes in LogManager now. - Jun Rao On May 2, 2014, 8:38 p.m., Timothy Chen wrote: --- This is an automatically generated e-mail. To reply, visit: https

Re: Review Request 20718: Patch for KAFKA-1384

2014-05-05 Thread Jun Rao
://reviews.apache.org/r/20718/#comment75904 This diagram is very helpful. We need an arrow from RunningAsController to RendingControlledShutdown. - Jun Rao On May 2, 2014, 7:13 p.m., Timothy Chen wrote: --- This is an automatically

Re: Review Request 20745: Patch for KAFKA-1397

2014-05-05 Thread Jun Rao
://reviews.apache.org/r/20745/#comment75940 The follower could be the controller. So, we probably should shutdown a replica that's not the controller. - Jun Rao On May 5, 2014, 6:17 p.m., Timothy Chen wrote

Re: Review Request 20718: Patch for KAFKA-1384

2014-05-05 Thread Jun Rao
,flushCheckMs: Long,flushCheckpointMs: Long,retentionCheckMs: Long,scheduler: kafka.utils.Scheduler,brokerState: kafka.server.BrokerState,time: kafka.utils.Time)kafka.log.LogManager. Unspecified value parameter brokerState. return new LogManager(logDirs, - Jun Rao On May 5, 2014, 6:05 p.m., Timothy

Re: Review Request 20718: Patch for KAFKA-1384

2014-05-05 Thread Jun Rao
parameters as we do in testRecoveryDirectoryMappingWithTrailingSlash()? core/src/test/scala/unit/kafka/log/LogManagerTest.scala https://reviews.apache.org/r/20718/#comment76010 Could we explicitly specify all parameters as we do in testRecoveryDirectoryMappingWithTrailingSlash()? - Jun Rao

Re: QOS on Producer Side

2014-05-06 Thread Jun Rao
will be very helpful. Thanks, Bhavesh On Mon, May 5, 2014 at 9:29 PM, Jun Rao jun...@gmail.com wrote: At LinkedIn, we only buffer messages in memory in the producer. We try as much as we can to make the Kafka cluster always available (with replication). The on failure feature you mentioned can

Re: Review Request 20718: Patch for KAFKA-1384

2014-05-06 Thread Jun Rao
/scala/kafka/log/Log.scala https://reviews.apache.org/r/20718/#comment76087 No need for this change. core/src/main/scala/kafka/server/KafkaServerStartable.scala https://reviews.apache.org/r/20718/#comment76088 Could you add a comment for the intent of this method? - Jun Rao On May 6

Re: Review Request 20997: Fix KAFKA-1432: address Neha's comments

2014-05-07 Thread Jun Rao
://reviews.apache.org/r/20997/#comment76155 Shouldn't we add the --new-producer option? - Jun Rao On May 5, 2014, 10:24 p.m., Guozhang Wang wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20997

Re: QOS on Producer Side

2014-05-07 Thread Jun Rao
this in a file.Is there an alternative solution ? What is best way to achieve QOS on producer side ? Also, will new Producer API will work with Broker version 0.8.0 ? Please let me know. Thanks, Bhavesh On Tue, May 6, 2014 at 8:46 AM, Jun Rao jun...@gmail.com wrote: The callback

test

2014-05-10 Thread Jun Rao
Some emails don't seem to be delivered to the mailing list. This is just a test to see if the mailing list still works. Jun

Re: Review Request 21239: Patch for KAFKA-1431

2014-05-11 Thread Jun Rao
://reviews.apache.org/r/21239/#comment76487 Could we change the description to sth like if specified, the consumer path in zookeeper is deleted when starting up. - Jun Rao On May 8, 2014, 9:58 p.m., Sriharsha Chintalapani wrote

Re: Review Request 21174: Fix KAFKA-1396

2014-05-11 Thread Jun Rao
return a valid leader (or an exception will be thrown), should we just return Int, instead of Option[Int]? - Jun Rao On May 9, 2014, 5:35 p.m., Guozhang Wang wrote: --- This is an automatically generated e-mail. To reply, visit: https

Re: Review Request 19731: Patch for KAFKA-1328

2014-05-12 Thread Jun Rao
/ConsumerConfig.java https://reviews.apache.org/r/19731/#comment76701 Do we need a config that allows the consumer to pick a strategy for dividing partitions among consumers? - Jun Rao On May 10, 2014, 12:18 a.m., Neha Narkhede wrote

Re: Review Request 19731: Patch for KAFKA-1328

2014-05-12 Thread Jun Rao
On May 7, 2014, 4:50 p.m., Jun Rao wrote: clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java, lines 87-88 https://reviews.apache.org/r/19731/diff/9/?file=574802#file574802line87 The Future thing doesn't work well in this case. This is because the caller

Review Request 21428: Patch for KAFKA-1450

2014-05-14 Thread Jun Rao
0b668f230c8556fdf08654ce522a11847d0bf39b core/src/test/scala/unit/kafka/utils/TestUtils.scala 4da0f2c245f75ff0dcab4ecf0af085ab9f8da1bb Diff: https://reviews.apache.org/r/21428/diff/ Testing --- Thanks, Jun Rao

Re: Review Request 21174: Fix KAFKA-1396

2014-05-14 Thread Jun Rao
the leader now. You will have to add it back. core/src/test/scala/integration/kafka/api/ProducerFailureHandlingTest.scala https://reviews.apache.org/r/21174/#comment76327 Could we use TestUtils.createNewProducer()? - Jun Rao On May 7, 2014, 9:56 p.m., Guozhang Wang wrote

Re: kafka performance question

2014-05-15 Thread Jun Rao
How many brokers and partitions do you have? You may try increasing batch.num.messages. Thanks, Jun On Tue, May 13, 2014 at 5:56 PM, Zhujie (zhujie, Smartcare) first.zhu...@huawei.com wrote: Dear all, We want to use kafka to collect and dispatch data file, but the performance is maybe

Re: Review Request 21174: Fix KAFKA-1396

2014-05-15 Thread Jun Rao
. */ - Jun Rao On May 8, 2014, 10:57 p.m., Guozhang Wang wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21174

Re: Review Request 19731: Patch for KAFKA-1328

2014-05-15 Thread Jun Rao
/KafkaConsumer.java https://reviews.apache.org/r/19731/#comment76177 The returned offset should be the next offset. See the comment on exposing nextOffset(). - Jun Rao On May 5, 2014, 6:35 p.m., Neha Narkhede wrote: --- This is an automatically

Re: Review Request 21239: Patch for KAFKA-1431

2014-05-16 Thread Jun Rao
://reviews.apache.org/r/21239/#comment77127 Since this is a command line tool, we should print the message to stdout, instead of logging, so that users can see the message on the console. - Jun Rao On May 14, 2014, 7:08 p.m., Sriharsha Chintalapani wrote

Re: Review Request 21398: Fix KAFKA-1445 v2

2014-05-16 Thread Jun Rao
/RecordAccumulator.java https://reviews.apache.org/r/21398/#comment77132 There could be ready partitions after their nodes are identified as ready. So we need to account for those. Probably the check for readyNodes can be moved to before line 215. - Jun Rao On May 14, 2014, 11:28 p.m

Re: Max Message Size issue

2014-05-16 Thread Jun Rao
You need to change max.message.size on the brokers. Thanks, Jun On Fri, May 16, 2014 at 11:02 AM, Bhavesh Mistry mistry.p.bhav...@gmail.com wrote: Hi Kafka Dev Group, We are using Kafka version 0.8 and I am getting following exception: WARN warn, Produce request with correlation id

Re: Make kafka storage engine pluggable and provide a HDFS plugin?

2014-05-19 Thread Jun Rao
Which version of Kafka are you using? Cluster expansion should be reliable in 0.8.1.1. As for using HDFS, we thought about it when implement replication in Kafka. The short answer is that using HDFS in Kafka is not easy. You can see the discussion in https://issues.apache.org/jira/browse/KAFKA-50

Re: Review Request 19731: Patch for KAFKA-1328

2014-05-19 Thread Jun Rao
/#comment77455 To be consistent with ConsumerRecord, we probably should just return Exception. clients/src/main/java/org/apache/kafka/clients/consumer/internals/FutureOffsetMetadata.java https://reviews.apache.org/r/19731/#comment77456 Do we still need this class? - Jun Rao On May 16, 2014

Re: Review Request 19731: Patch for KAFKA-1328

2014-05-19 Thread Jun Rao
/ConsumerConfig.java https://reviews.apache.org/r/19731/#comment77485 Have we added the config for selecting the strategy for assigning partitions? - Jun Rao On May 16, 2014, 6:46 p.m., Neha Narkhede wrote: --- This is an automatically

Re: Review Request 21588: Fix KAFKA-1430

2014-05-20 Thread Jun Rao
/kafka/server/RequestPurgatory.scala https://reviews.apache.org/r/21588/#comment77612 The purgeSatisfied() here is redundant since it's already done in purgeSatisfied(). core/src/main/scala/kafka/server/RequestPurgatory.scala https://reviews.apache.org/r/21588/#comment77559 - Jun

Re: Review Request 19731: Patch for KAFKA-1328

2014-05-20 Thread Jun Rao
/Consumer.java https://reviews.apache.org/r/19731/#comment77768 Could we add in the comment that the timeout is in ms? - Jun Rao On May 20, 2014, 11:34 p.m., Neha Narkhede wrote: --- This is an automatically generated e-mail

Re: Review Request 21744: Patch for KAFKA-1446

2014-05-21 Thread Jun Rao
/ZookeeperConsumerConnector.scala https://reviews.apache.org/r/21744/#comment77895 Timer is basically a histogram of the duration of a type of event and a meter of the rate of its occurrence. So, we don't need a separate meter to measure the rate. - Jun Rao On May 21, 2014, 12:48 a.m., Sriharsha Chintalapani wrote

Re: Review Request 21663: Patch for KAFKA-1456

2014-05-23 Thread Jun Rao
in ProducerCompressionTest and MessageTest too? - Jun Rao On May 20, 2014, 6:24 a.m., James Oliver wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21663

Re: Regarding Kafka message pushing

2014-05-26 Thread Jun Rao
Which version of Kafka are you using? The log before FailedToSendMessage should tell you the cause. Could you paste that? Thanks, Jun On Sun, May 25, 2014 at 10:59 PM, SAURABH VERMA saurabhdec1...@gmail.comwrote: Hello, I am experimenting with Kafka. Let me explain in brief what I am

Re: Simple Kafka producer in Windows does not run

2014-05-26 Thread Jun Rao
Which version of Kafka are you using? The log before FailedToSendMessage should tell you the cause. Could you paste that? Thanks, Jun On Mon, May 26, 2014 at 6:01 AM, Srivatsan Vengada Nallazhagappan srivatsanvengad...@infosys.com wrote: Hi, The program given below is not running in

Re: Review Request 21878: KAFKA-1468: Misc. improvements to test scripts from benchmarking

2014-05-27 Thread Jun Rao
/tools/ProducerPerformance.java https://reviews.apache.org/r/21878/#comment78417 Should we handle the error case here? - Jun Rao On May 23, 2014, 9:08 p.m., Jay Kreps wrote: --- This is an automatically generated e-mail. To reply, visit

Re: Security mechanism (kerberos) in Kafka

2014-05-29 Thread Jun Rao
Security is on our to-do list. Our current plan is to complete the new consumer first in the next 2-3 months. We probably can take on the security feature afterwards. Thanks, Jun On Thu, May 29, 2014 at 7:37 AM, Pogontsev, Roman roman.pogont...@morganstanley.com wrote: Hi, Our team has a

Re: Review Request 21878: Patch for KAFKA-1468

2014-05-29 Thread Jun Rao
/ProducerPerformance.java https://reviews.apache.org/r/21878/#comment78606 Perhaps we can rename target_throughput to target_records_per_sec? - Jun Rao On May 28, 2014, 11:15 p.m., Jay Kreps wrote: --- This is an automatically

Re: Producer Side Metric Details

2014-05-29 Thread Jun Rao
topics ? The numbers for AllTopics* are same as per topoic (logmon.error-*) ? Can you guys please clarify this ?Do you guys intended to work as global aggregated for ALL topic ? Please see the following image: Thanks, Bhavesh On Wed, May 28, 2014 at 9:44 PM, Jun Rao jun

Re: Producer Side Metric Details

2014-05-30 Thread Jun Rao
, Jun Rao jun...@gmail.com wrote: You need to configure each producer with a different client.id. Thanks, Jun On Thu, May 29, 2014 at 5:38 PM, Bhavesh Mistry mistry.p.bhav...@gmail.com wrote: We have 4 producers (for 4 topics) within one JVM and all of them configured

Re: Build failed in Jenkins: Kafka-trunk #197

2014-06-03 Thread Jun Rao
console consumer properties via a single --property command line parameter;; patched by Sriharsha Chintalapani; reviewed by Jun Rao -- [...truncated 1648 lines...] at kafka.integration.KafkaServerTestHarness$class.setUp

Re: Review Request 22063: Patch for KAFKA-1472

2014-06-05 Thread Jun Rao
/Sender.java https://reviews.apache.org/r/22063/#comment79363 Could we add a global compression-rate metric too? - Jun Rao On June 4, 2014, 5:31 p.m., Dong Lin wrote: --- This is an automatically generated e-mail. To reply

Re: Review Request 21937: Patch for KAFKA-1316

2014-06-06 Thread Jun Rao
this is called in the producer or sender. - Jun Rao On June 3, 2014, 9:33 p.m., Jay Kreps wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21937

Re: Review Request 21588: Fix KAFKA-1430, round two

2014-06-06 Thread Jun Rao
to override val? core/src/main/scala/kafka/server/RequestPurgatory.scala https://reviews.apache.org/r/21588/#comment79645 We will need to sync on t when doing the check (same as in line 187) to avoid the race condition that can cause two responses to be sent for the same request. - Jun Rao

Re: Review Request 21899: Patch for KAFKA-1382

2014-06-09 Thread Jun Rao
. - Jun Rao On June 7, 2014, 4 p.m., Sriharsha Chintalapani wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21899/ --- (Updated

Re: Review Request 21899: Patch for KAFKA-1382

2014-06-10 Thread Jun Rao
the optionalChecker to further check if the write indeed have succeeded. core/src/test/scala/unit/kafka/utils/ReplicationUtilsTest.scala https://reviews.apache.org/r/21899/#comment79984 If we use the default equal for leaderAndIsr, do we still need this test? - Jun Rao On June 10, 2014, 1:23 a.m

Review Request 22446: Patch for KAFKA-1488

2014-06-10 Thread Jun Rao
/diff/ Testing --- Thanks, Jun Rao

Re: Review Request 22446: Patch for KAFKA-1488

2014-06-10 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22446/#review45314 --- On June 10, 2014, 11:15 p.m., Jun Rao wrote: --- This is an automatically

Re: Review Request 21899: Patch for KAFKA-1382

2014-06-12 Thread Jun Rao
Could we rename newLeader to expectedLeader? core/src/main/scala/kafka/utils/ZkUtils.scala https://reviews.apache.org/r/21899/#comment80351 the write indeed have = the previous write has indeed - Jun Rao On June 11, 2014, 4:37 p.m., Sriharsha Chintalapani wrote

Re: Regarding the Producers and Consumers of Kafka

2014-06-16 Thread Jun Rao
Kafka topics are auto created by default. A consumer can specify a regex .* if it wants to consumer all topics. Thanks, Jun On Mon, Jun 16, 2014 at 4:35 PM, Chiran Jain ichiranj...@gmail.com wrote: Query: Making an interface of router as the producer of kafka cluster. Issue: My router's

Re: Review Request 22802: KAFKA-1308, publish test jars

2014-06-20 Thread Jun Rao
test jar created for publish. - Jun Rao On June 20, 2014, 5:27 a.m., Jakob Homan wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22802

<    1   2   3   4   5   6   7   8   9   10   >