Re: Review Request 21865: Patch for KAFKA-1438
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/ --- (Updated May 30, 2014, 6:35 p.m.) Review request for kafka. Bugs: KAFKA-1438 https://issues.apache.org/jira/browse/KAFKA-1438 Repository: kafka Description (updated) --- KAFKA-1438. Migrate kafka client tools. Diffs (updated) - bin/kafka-console-consumer.sh e410dde271eea996416f7a67561e45b78f8e8a69 bin/kafka-console-producer.sh cd8ce62fe51e3ad64908146f727f73fc6e35b35e bin/kafka-consumer-perf-test.sh 4ed3ed9fb12edf00a82c06c4e59270a60df43ecd bin/kafka-producer-perf-test.sh b4efc29d76ff8f53a310f812b9ed3de6875465b5 bin/kafka-run-class.sh d2fc8c0e413769bb89ada99790b90ad616044f28 bin/kafka-simple-consumer-perf-test.sh 2d3e3d3be21065a4d446b9f8ed2e9d31c0685456 bin/windows/kafka-run-class.bat f4d2904a3320ae756ed4171f6e99566f1f0cf963 core/src/main/scala/kafka/consumer/ConsoleConsumer.scala 1a16c691683dda0c53f316e3c4797ea38e776574 core/src/main/scala/kafka/producer/ConsoleProducer.scala a2af988d99a94a20291d6a2dc9bec73197f1b756 core/src/main/scala/kafka/server/OffsetManager.scala 54176283e5f199d9db08e2181e7ddfbb9c801fab perf/config/log4j.properties 542b7391550b2a8bb091a38d08583bea9b3111b6 perf/src/main/scala/kafka/perf/ConsumerPerformance.scala 4dde4687bada1202eaf54be2b06c2156e7726b23 perf/src/main/scala/kafka/perf/PerfConfig.scala a8fc6b9ec81c49c73c35172ddc5a67c6a96c799a perf/src/main/scala/kafka/perf/ProducerPerformance.scala 00fa90bb286900fb33a05f55e3c34c2cbb5620a1 perf/src/main/scala/kafka/perf/SimpleConsumerPerformance.scala c52ada0a30ae49c8d576703859b9d49348f62feb system_test/broker_failure/bin/run-test.sh 1f11180e33974099744ca1fbaefbd1047675823e system_test/producer_perf/bin/run-compression-test.sh ea20f0dbd803ebaa7eb1b058d5153db10dca7ca7 system_test/producer_perf/bin/run-test.sh bb60817edda2bc2de955b4a66ca0f44c08493a03 system_test/utils/kafka_system_test_utils.py de02e471c4efd688f3915aa5854cc4b29860050d Diff: https://reviews.apache.org/r/21865/diff/ Testing --- Thanks, Sriharsha Chintalapani
Re: Review Request 21865: Patch for KAFKA-1438
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/ --- (Updated May 30, 2014, 6:38 p.m.) Review request for kafka. Bugs: KAFKA-1438 https://issues.apache.org/jira/browse/KAFKA-1438 Repository: kafka Description (updated) --- KAFKA-1438. Migrate kafka client tools. KAFKA-1438. Migrate kafka client tools. Diffs (updated) - bin/kafka-console-consumer.sh e410dde271eea996416f7a67561e45b78f8e8a69 bin/kafka-console-producer.sh cd8ce62fe51e3ad64908146f727f73fc6e35b35e bin/kafka-consumer-perf-test.sh 4ed3ed9fb12edf00a82c06c4e59270a60df43ecd bin/kafka-producer-perf-test.sh b4efc29d76ff8f53a310f812b9ed3de6875465b5 bin/kafka-run-class.sh d2fc8c0e413769bb89ada99790b90ad616044f28 bin/kafka-simple-consumer-perf-test.sh 2d3e3d3be21065a4d446b9f8ed2e9d31c0685456 core/src/main/scala/kafka/consumer/ConsoleConsumer.scala 1a16c691683dda0c53f316e3c4797ea38e776574 core/src/main/scala/kafka/producer/ConsoleProducer.scala a2af988d99a94a20291d6a2dc9bec73197f1b756 core/src/main/scala/kafka/server/OffsetManager.scala 54176283e5f199d9db08e2181e7ddfbb9c801fab perf/config/log4j.properties 542b7391550b2a8bb091a38d08583bea9b3111b6 perf/src/main/scala/kafka/perf/ConsumerPerformance.scala 4dde4687bada1202eaf54be2b06c2156e7726b23 perf/src/main/scala/kafka/perf/PerfConfig.scala a8fc6b9ec81c49c73c35172ddc5a67c6a96c799a perf/src/main/scala/kafka/perf/ProducerPerformance.scala 00fa90bb286900fb33a05f55e3c34c2cbb5620a1 perf/src/main/scala/kafka/perf/SimpleConsumerPerformance.scala c52ada0a30ae49c8d576703859b9d49348f62feb system_test/broker_failure/bin/run-test.sh 1f11180e33974099744ca1fbaefbd1047675823e system_test/producer_perf/bin/run-compression-test.sh ea20f0dbd803ebaa7eb1b058d5153db10dca7ca7 system_test/producer_perf/bin/run-test.sh bb60817edda2bc2de955b4a66ca0f44c08493a03 system_test/utils/kafka_system_test_utils.py de02e471c4efd688f3915aa5854cc4b29860050d Diff: https://reviews.apache.org/r/21865/diff/ Testing --- Thanks, Sriharsha Chintalapani
Re: Review Request 21865: Patch for KAFKA-1438
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/ --- (Updated May 30, 2014, 6:42 p.m.) Review request for kafka. Bugs: KAFKA-1438 https://issues.apache.org/jira/browse/KAFKA-1438 Repository: kafka Description --- KAFKA-1438. Migrate kafka client tools. KAFKA-1438. Migrate kafka client tools. Diffs (updated) - bin/kafka-console-consumer.sh e410dde271eea996416f7a67561e45b78f8e8a69 bin/kafka-console-producer.sh cd8ce62fe51e3ad64908146f727f73fc6e35b35e bin/kafka-consumer-perf-test.sh 4ed3ed9fb12edf00a82c06c4e59270a60df43ecd bin/kafka-producer-perf-test.sh b4efc29d76ff8f53a310f812b9ed3de6875465b5 bin/kafka-run-class.sh d2fc8c0e413769bb89ada99790b90ad616044f28 bin/kafka-simple-consumer-perf-test.sh 2d3e3d3be21065a4d446b9f8ed2e9d31c0685456 core/src/main/scala/kafka/consumer/ConsoleConsumer.scala 1a16c691683dda0c53f316e3c4797ea38e776574 core/src/main/scala/kafka/producer/ConsoleProducer.scala a2af988d99a94a20291d6a2dc9bec73197f1b756 core/src/main/scala/kafka/server/OffsetManager.scala 54176283e5f199d9db08e2181e7ddfbb9c801fab perf/config/log4j.properties 542b7391550b2a8bb091a38d08583bea9b3111b6 perf/src/main/scala/kafka/perf/ConsumerPerformance.scala 4dde4687bada1202eaf54be2b06c2156e7726b23 perf/src/main/scala/kafka/perf/PerfConfig.scala a8fc6b9ec81c49c73c35172ddc5a67c6a96c799a perf/src/main/scala/kafka/perf/ProducerPerformance.scala 00fa90bb286900fb33a05f55e3c34c2cbb5620a1 perf/src/main/scala/kafka/perf/SimpleConsumerPerformance.scala c52ada0a30ae49c8d576703859b9d49348f62feb system_test/broker_failure/bin/run-test.sh 1f11180e33974099744ca1fbaefbd1047675823e system_test/producer_perf/bin/run-compression-test.sh ea20f0dbd803ebaa7eb1b058d5153db10dca7ca7 system_test/producer_perf/bin/run-test.sh bb60817edda2bc2de955b4a66ca0f44c08493a03 system_test/utils/kafka_system_test_utils.py de02e471c4efd688f3915aa5854cc4b29860050d Diff: https://reviews.apache.org/r/21865/diff/ Testing --- Thanks, Sriharsha Chintalapani
Re: Review Request 21865: Patch for KAFKA-1438
On May 28, 2014, 12:27 a.m., Guozhang Wang wrote: I get the same error as Neha on kafka-run-class.sh, could you double check if there is anything wrong with that diff? I did a git diff against origin/trunk and applied as a patch but the one uploaded to reviewboard having issue. I'll check it. Thanks - Sriharsha --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/#review44081 --- On May 28, 2014, 12:08 a.m., Sriharsha Chintalapani wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/ --- (Updated May 28, 2014, 12:08 a.m.) Review request for kafka. Bugs: KAFKA-1438 https://issues.apache.org/jira/browse/KAFKA-1438 Repository: kafka Description --- KAFKA-1438. Migrate Kafka Client tools. Diffs - bin/kafka-console-consumer.sh e410dde271eea996416f7a67561e45b78f8e8a69 bin/kafka-console-producer.sh cd8ce62fe51e3ad64908146f727f73fc6e35b35e bin/kafka-consumer-perf-test.sh 4ed3ed9fb12edf00a82c06c4e59270a60df43ecd bin/kafka-producer-perf-test.sh b4efc29d76ff8f53a310f812b9ed3de6875465b5 bin/kafka-run-class.sh d2fc8c0e413769bb89ada99790b90ad616044f28 bin/kafka-simple-consumer-perf-test.sh 2d3e3d3be21065a4d446b9f8ed2e9d31c0685456 bin/windows/kafka-console-consumer.bat 94b20a4d1d2d043b4c4280d00b807d1d27089ac1 bin/windows/kafka-console-producer.bat b116e64cc4d8b19e0038f3dd1ff41bc5eb47abab bin/windows/kafka-run-class.bat f4d2904a3320ae756ed4171f6e99566f1f0cf963 core/src/main/scala/kafka/consumer/ConsoleConsumer.scala 1a16c691683dda0c53f316e3c4797ea38e776574 core/src/main/scala/kafka/producer/ConsoleProducer.scala a2af988d99a94a20291d6a2dc9bec73197f1b756 core/src/main/scala/kafka/server/OffsetManager.scala 54176283e5f199d9db08e2181e7ddfbb9c801fab perf/config/log4j.properties 542b7391550b2a8bb091a38d08583bea9b3111b6 perf/src/main/scala/kafka/perf/ConsumerPerformance.scala 4dde4687bada1202eaf54be2b06c2156e7726b23 perf/src/main/scala/kafka/perf/PerfConfig.scala a8fc6b9ec81c49c73c35172ddc5a67c6a96c799a perf/src/main/scala/kafka/perf/ProducerPerformance.scala 00fa90bb286900fb33a05f55e3c34c2cbb5620a1 perf/src/main/scala/kafka/perf/SimpleConsumerPerformance.scala c52ada0a30ae49c8d576703859b9d49348f62feb Diff: https://reviews.apache.org/r/21865/diff/ Testing --- Thanks, Sriharsha Chintalapani
Re: Review Request 21865: Patch for KAFKA-1438
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/ --- (Updated May 28, 2014, 3:40 p.m.) Review request for kafka. Bugs: KAFKA-1438 https://issues.apache.org/jira/browse/KAFKA-1438 Repository: kafka Description (updated) --- KAFKA-1438. Migrate Kafka Client tools. KAFKA-1438. Migrate Kafka Client tools. Diffs (updated) - bin/kafka-console-consumer.sh e410dde271eea996416f7a67561e45b78f8e8a69 bin/kafka-console-producer.sh cd8ce62fe51e3ad64908146f727f73fc6e35b35e bin/kafka-consumer-perf-test.sh 4ed3ed9fb12edf00a82c06c4e59270a60df43ecd bin/kafka-producer-perf-test.sh b4efc29d76ff8f53a310f812b9ed3de6875465b5 bin/kafka-run-class.sh d2fc8c0e413769bb89ada99790b90ad616044f28 bin/kafka-simple-consumer-perf-test.sh 2d3e3d3be21065a4d446b9f8ed2e9d31c0685456 bin/windows/kafka-console-consumer.bat 94b20a4d1d2d043b4c4280d00b807d1d27089ac1 bin/windows/kafka-console-producer.bat b116e64cc4d8b19e0038f3dd1ff41bc5eb47abab core/src/main/scala/kafka/consumer/ConsoleConsumer.scala 1a16c691683dda0c53f316e3c4797ea38e776574 core/src/main/scala/kafka/producer/ConsoleProducer.scala a2af988d99a94a20291d6a2dc9bec73197f1b756 core/src/main/scala/kafka/server/OffsetManager.scala 54176283e5f199d9db08e2181e7ddfbb9c801fab perf/config/log4j.properties 542b7391550b2a8bb091a38d08583bea9b3111b6 perf/src/main/scala/kafka/perf/ConsumerPerformance.scala 4dde4687bada1202eaf54be2b06c2156e7726b23 perf/src/main/scala/kafka/perf/PerfConfig.scala a8fc6b9ec81c49c73c35172ddc5a67c6a96c799a perf/src/main/scala/kafka/perf/ProducerPerformance.scala 00fa90bb286900fb33a05f55e3c34c2cbb5620a1 perf/src/main/scala/kafka/perf/SimpleConsumerPerformance.scala c52ada0a30ae49c8d576703859b9d49348f62feb Diff: https://reviews.apache.org/r/21865/diff/ Testing --- Thanks, Sriharsha Chintalapani
Re: Review Request 21865: Patch for KAFKA-1438
On May 28, 2014, 12:27 a.m., Guozhang Wang wrote: I get the same error as Neha on kafka-run-class.sh, could you double check if there is anything wrong with that diff? Sriharsha Chintalapani wrote: I did a git diff against origin/trunk and applied as a patch but the one uploaded to reviewboard having issue. I'll check it. Thanks I am not sure what exactly going wrong with the uploaded patch. I tried with creating a local patch with git diff that works fine only when uploaded review board the patch is not working. I undid changes bin/windows/kafka-run-class.bat and uploaded a patch this works fine. If it makes it easier can you remove adding perf dependencies part of class path in kafka-run-class.bat as part of this fix. Thanks - Sriharsha --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/#review44081 --- On May 28, 2014, 3:40 p.m., Sriharsha Chintalapani wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/ --- (Updated May 28, 2014, 3:40 p.m.) Review request for kafka. Bugs: KAFKA-1438 https://issues.apache.org/jira/browse/KAFKA-1438 Repository: kafka Description --- KAFKA-1438. Migrate Kafka Client tools. KAFKA-1438. Migrate Kafka Client tools. Diffs - bin/kafka-console-consumer.sh e410dde271eea996416f7a67561e45b78f8e8a69 bin/kafka-console-producer.sh cd8ce62fe51e3ad64908146f727f73fc6e35b35e bin/kafka-consumer-perf-test.sh 4ed3ed9fb12edf00a82c06c4e59270a60df43ecd bin/kafka-producer-perf-test.sh b4efc29d76ff8f53a310f812b9ed3de6875465b5 bin/kafka-run-class.sh d2fc8c0e413769bb89ada99790b90ad616044f28 bin/kafka-simple-consumer-perf-test.sh 2d3e3d3be21065a4d446b9f8ed2e9d31c0685456 bin/windows/kafka-console-consumer.bat 94b20a4d1d2d043b4c4280d00b807d1d27089ac1 bin/windows/kafka-console-producer.bat b116e64cc4d8b19e0038f3dd1ff41bc5eb47abab core/src/main/scala/kafka/consumer/ConsoleConsumer.scala 1a16c691683dda0c53f316e3c4797ea38e776574 core/src/main/scala/kafka/producer/ConsoleProducer.scala a2af988d99a94a20291d6a2dc9bec73197f1b756 core/src/main/scala/kafka/server/OffsetManager.scala 54176283e5f199d9db08e2181e7ddfbb9c801fab perf/config/log4j.properties 542b7391550b2a8bb091a38d08583bea9b3111b6 perf/src/main/scala/kafka/perf/ConsumerPerformance.scala 4dde4687bada1202eaf54be2b06c2156e7726b23 perf/src/main/scala/kafka/perf/PerfConfig.scala a8fc6b9ec81c49c73c35172ddc5a67c6a96c799a perf/src/main/scala/kafka/perf/ProducerPerformance.scala 00fa90bb286900fb33a05f55e3c34c2cbb5620a1 perf/src/main/scala/kafka/perf/SimpleConsumerPerformance.scala c52ada0a30ae49c8d576703859b9d49348f62feb Diff: https://reviews.apache.org/r/21865/diff/ Testing --- Thanks, Sriharsha Chintalapani
Re: Review Request 21865: Patch for KAFKA-1438
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/#review43988 --- We also want to move the *Performance out of perf into tools and remove the perf subproject config/tools-log4j.properties https://reviews.apache.org/r/21865/#comment78228 Why do we need this? perf/src/main/scala/kafka/perf/ConsumerPerformance.scala https://reviews.apache.org/r/21865/#comment78227 Why not let the getClass() remain? perf/src/main/scala/kafka/perf/ProducerPerformance.scala https://reviews.apache.org/r/21865/#comment78229 same here - Neha Narkhede On May 23, 2014, 3:52 p.m., Sriharsha Chintalapani wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/ --- (Updated May 23, 2014, 3:52 p.m.) Review request for kafka. Bugs: KAFKA-1438 https://issues.apache.org/jira/browse/KAFKA-1438 Repository: kafka Description --- KAFKA-1438. Migrate Kafka Client tools. Diffs - bin/kafka-console-consumer.sh e410dde271eea996416f7a67561e45b78f8e8a69 bin/kafka-consumer-perf-test.sh 4ed3ed9fb12edf00a82c06c4e59270a60df43ecd bin/kafka-producer-perf-test.sh b4efc29d76ff8f53a310f812b9ed3de6875465b5 bin/kafka-run-class.sh d2fc8c0e413769bb89ada99790b90ad616044f28 bin/kafka-simple-consumer-perf-test.sh 2d3e3d3be21065a4d446b9f8ed2e9d31c0685456 bin/windows/kafka-console-consumer.bat 94b20a4d1d2d043b4c4280d00b807d1d27089ac1 bin/windows/kafka-run-class.bat f4d2904a3320ae756ed4171f6e99566f1f0cf963 config/tools-log4j.properties 52f07c96019b4083fc78f62cfb0a81080327e847 core/src/main/scala/kafka/consumer/ConsoleConsumer.scala 24c92877cc7df20b82b7a438d39120b1caa3924d core/src/main/scala/kafka/server/OffsetManager.scala 54176283e5f199d9db08e2181e7ddfbb9c801fab perf/config/log4j.properties 542b7391550b2a8bb091a38d08583bea9b3111b6 perf/src/main/scala/kafka/perf/ConsumerPerformance.scala 4dde4687bada1202eaf54be2b06c2156e7726b23 perf/src/main/scala/kafka/perf/PerfConfig.scala a8fc6b9ec81c49c73c35172ddc5a67c6a96c799a perf/src/main/scala/kafka/perf/ProducerPerformance.scala 00fa90bb286900fb33a05f55e3c34c2cbb5620a1 perf/src/main/scala/kafka/perf/SimpleConsumerPerformance.scala c52ada0a30ae49c8d576703859b9d49348f62feb Diff: https://reviews.apache.org/r/21865/diff/ Testing --- Thanks, Sriharsha Chintalapani
Re: Review Request 21865: Patch for KAFKA-1438
On May 27, 2014, 5:26 p.m., Neha Narkhede wrote: perf/src/main/scala/kafka/perf/ProducerPerformance.scala, line 42 https://reviews.apache.org/r/21865/diff/1/?file=589809#file589809line42 same here current perf project has its own log4j.properties under perf/config and it writes to perf.log . I thought users expects the logs goto perf.log and tools-log4j.properties being included in the classpath by kafka-run-class.sh. I added specific logging properties for perf related classes and using that by referring to kafka.tools.performance. By default tools-log4j uses ConsoleAppender and this config used by other tools to write error messages to the console. - Sriharsha --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/#review43988 --- On May 23, 2014, 3:52 p.m., Sriharsha Chintalapani wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/ --- (Updated May 23, 2014, 3:52 p.m.) Review request for kafka. Bugs: KAFKA-1438 https://issues.apache.org/jira/browse/KAFKA-1438 Repository: kafka Description --- KAFKA-1438. Migrate Kafka Client tools. Diffs - bin/kafka-console-consumer.sh e410dde271eea996416f7a67561e45b78f8e8a69 bin/kafka-consumer-perf-test.sh 4ed3ed9fb12edf00a82c06c4e59270a60df43ecd bin/kafka-producer-perf-test.sh b4efc29d76ff8f53a310f812b9ed3de6875465b5 bin/kafka-run-class.sh d2fc8c0e413769bb89ada99790b90ad616044f28 bin/kafka-simple-consumer-perf-test.sh 2d3e3d3be21065a4d446b9f8ed2e9d31c0685456 bin/windows/kafka-console-consumer.bat 94b20a4d1d2d043b4c4280d00b807d1d27089ac1 bin/windows/kafka-run-class.bat f4d2904a3320ae756ed4171f6e99566f1f0cf963 config/tools-log4j.properties 52f07c96019b4083fc78f62cfb0a81080327e847 core/src/main/scala/kafka/consumer/ConsoleConsumer.scala 24c92877cc7df20b82b7a438d39120b1caa3924d core/src/main/scala/kafka/server/OffsetManager.scala 54176283e5f199d9db08e2181e7ddfbb9c801fab perf/config/log4j.properties 542b7391550b2a8bb091a38d08583bea9b3111b6 perf/src/main/scala/kafka/perf/ConsumerPerformance.scala 4dde4687bada1202eaf54be2b06c2156e7726b23 perf/src/main/scala/kafka/perf/PerfConfig.scala a8fc6b9ec81c49c73c35172ddc5a67c6a96c799a perf/src/main/scala/kafka/perf/ProducerPerformance.scala 00fa90bb286900fb33a05f55e3c34c2cbb5620a1 perf/src/main/scala/kafka/perf/SimpleConsumerPerformance.scala c52ada0a30ae49c8d576703859b9d49348f62feb Diff: https://reviews.apache.org/r/21865/diff/ Testing --- Thanks, Sriharsha Chintalapani
Re: Review Request 21865: Patch for KAFKA-1438
On May 27, 2014, 5:26 p.m., Neha Narkhede wrote: We also want to move the *Performance out of perf into tools and remove the perf subproject The diff contains changes necessary for moving perf classes under tools. - Sriharsha --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/#review43988 --- On May 23, 2014, 3:52 p.m., Sriharsha Chintalapani wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/ --- (Updated May 23, 2014, 3:52 p.m.) Review request for kafka. Bugs: KAFKA-1438 https://issues.apache.org/jira/browse/KAFKA-1438 Repository: kafka Description --- KAFKA-1438. Migrate Kafka Client tools. Diffs - bin/kafka-console-consumer.sh e410dde271eea996416f7a67561e45b78f8e8a69 bin/kafka-consumer-perf-test.sh 4ed3ed9fb12edf00a82c06c4e59270a60df43ecd bin/kafka-producer-perf-test.sh b4efc29d76ff8f53a310f812b9ed3de6875465b5 bin/kafka-run-class.sh d2fc8c0e413769bb89ada99790b90ad616044f28 bin/kafka-simple-consumer-perf-test.sh 2d3e3d3be21065a4d446b9f8ed2e9d31c0685456 bin/windows/kafka-console-consumer.bat 94b20a4d1d2d043b4c4280d00b807d1d27089ac1 bin/windows/kafka-run-class.bat f4d2904a3320ae756ed4171f6e99566f1f0cf963 config/tools-log4j.properties 52f07c96019b4083fc78f62cfb0a81080327e847 core/src/main/scala/kafka/consumer/ConsoleConsumer.scala 24c92877cc7df20b82b7a438d39120b1caa3924d core/src/main/scala/kafka/server/OffsetManager.scala 54176283e5f199d9db08e2181e7ddfbb9c801fab perf/config/log4j.properties 542b7391550b2a8bb091a38d08583bea9b3111b6 perf/src/main/scala/kafka/perf/ConsumerPerformance.scala 4dde4687bada1202eaf54be2b06c2156e7726b23 perf/src/main/scala/kafka/perf/PerfConfig.scala a8fc6b9ec81c49c73c35172ddc5a67c6a96c799a perf/src/main/scala/kafka/perf/ProducerPerformance.scala 00fa90bb286900fb33a05f55e3c34c2cbb5620a1 perf/src/main/scala/kafka/perf/SimpleConsumerPerformance.scala c52ada0a30ae49c8d576703859b9d49348f62feb Diff: https://reviews.apache.org/r/21865/diff/ Testing --- Thanks, Sriharsha Chintalapani
Re: Review Request 21865: Patch for KAFKA-1438
On May 27, 2014, 5:26 p.m., Neha Narkhede wrote: We also want to move the *Performance out of perf into tools and remove the perf subproject Sriharsha Chintalapani wrote: The diff contains changes necessary for moving perf classes under tools. I applied the patch. It failed to apply on the .bat file and didn't get rid of the perf subproject. Would you mind uploading an updated patch? On May 27, 2014, 5:26 p.m., Neha Narkhede wrote: perf/src/main/scala/kafka/perf/ProducerPerformance.scala, line 42 https://reviews.apache.org/r/21865/diff/1/?file=589809#file589809line42 same here Sriharsha Chintalapani wrote: current perf project has its own log4j.properties under perf/config and it writes to perf.log . I thought users expects the logs goto perf.log and tools-log4j.properties being included in the classpath by kafka-run-class.sh. I added specific logging properties for perf related classes and using that by referring to kafka.tools.performance. By default tools-log4j uses ConsoleAppender and this config used by other tools to write error messages to the console. I guess it will be convenient to have the same ConsoleAppender behavior for perf tools as well. Users can pipe the output to other files as they see fit right? - Neha --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/#review43988 --- On May 23, 2014, 3:52 p.m., Sriharsha Chintalapani wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/ --- (Updated May 23, 2014, 3:52 p.m.) Review request for kafka. Bugs: KAFKA-1438 https://issues.apache.org/jira/browse/KAFKA-1438 Repository: kafka Description --- KAFKA-1438. Migrate Kafka Client tools. Diffs - bin/kafka-console-consumer.sh e410dde271eea996416f7a67561e45b78f8e8a69 bin/kafka-consumer-perf-test.sh 4ed3ed9fb12edf00a82c06c4e59270a60df43ecd bin/kafka-producer-perf-test.sh b4efc29d76ff8f53a310f812b9ed3de6875465b5 bin/kafka-run-class.sh d2fc8c0e413769bb89ada99790b90ad616044f28 bin/kafka-simple-consumer-perf-test.sh 2d3e3d3be21065a4d446b9f8ed2e9d31c0685456 bin/windows/kafka-console-consumer.bat 94b20a4d1d2d043b4c4280d00b807d1d27089ac1 bin/windows/kafka-run-class.bat f4d2904a3320ae756ed4171f6e99566f1f0cf963 config/tools-log4j.properties 52f07c96019b4083fc78f62cfb0a81080327e847 core/src/main/scala/kafka/consumer/ConsoleConsumer.scala 24c92877cc7df20b82b7a438d39120b1caa3924d core/src/main/scala/kafka/server/OffsetManager.scala 54176283e5f199d9db08e2181e7ddfbb9c801fab perf/config/log4j.properties 542b7391550b2a8bb091a38d08583bea9b3111b6 perf/src/main/scala/kafka/perf/ConsumerPerformance.scala 4dde4687bada1202eaf54be2b06c2156e7726b23 perf/src/main/scala/kafka/perf/PerfConfig.scala a8fc6b9ec81c49c73c35172ddc5a67c6a96c799a perf/src/main/scala/kafka/perf/ProducerPerformance.scala 00fa90bb286900fb33a05f55e3c34c2cbb5620a1 perf/src/main/scala/kafka/perf/SimpleConsumerPerformance.scala c52ada0a30ae49c8d576703859b9d49348f62feb Diff: https://reviews.apache.org/r/21865/diff/ Testing --- Thanks, Sriharsha Chintalapani
Re: Review Request 21865: Patch for KAFKA-1438
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/ --- (Updated May 27, 2014, 6:45 p.m.) Review request for kafka. Bugs: KAFKA-1438 https://issues.apache.org/jira/browse/KAFKA-1438 Repository: kafka Description --- KAFKA-1438. Migrate Kafka Client tools. Diffs (updated) - bin/kafka-console-consumer.sh e410dde271eea996416f7a67561e45b78f8e8a69 bin/kafka-consumer-perf-test.sh 4ed3ed9fb12edf00a82c06c4e59270a60df43ecd bin/kafka-producer-perf-test.sh b4efc29d76ff8f53a310f812b9ed3de6875465b5 bin/kafka-run-class.sh d2fc8c0e413769bb89ada99790b90ad616044f28 bin/kafka-simple-consumer-perf-test.sh 2d3e3d3be21065a4d446b9f8ed2e9d31c0685456 bin/windows/kafka-console-consumer.bat 94b20a4d1d2d043b4c4280d00b807d1d27089ac1 bin/windows/kafka-run-class.bat f4d2904a3320ae756ed4171f6e99566f1f0cf963 config/tools-log4j.properties 52f07c96019b4083fc78f62cfb0a81080327e847 core/src/main/scala/kafka/consumer/ConsoleConsumer.scala 24c92877cc7df20b82b7a438d39120b1caa3924d core/src/main/scala/kafka/server/OffsetManager.scala 54176283e5f199d9db08e2181e7ddfbb9c801fab perf/config/log4j.properties 542b7391550b2a8bb091a38d08583bea9b3111b6 perf/src/main/scala/kafka/perf/ConsumerPerformance.scala 4dde4687bada1202eaf54be2b06c2156e7726b23 perf/src/main/scala/kafka/perf/PerfConfig.scala a8fc6b9ec81c49c73c35172ddc5a67c6a96c799a perf/src/main/scala/kafka/perf/ProducerPerformance.scala 00fa90bb286900fb33a05f55e3c34c2cbb5620a1 perf/src/main/scala/kafka/perf/SimpleConsumerPerformance.scala c52ada0a30ae49c8d576703859b9d49348f62feb Diff: https://reviews.apache.org/r/21865/diff/ Testing --- Thanks, Sriharsha Chintalapani
Re: Review Request 21865: Patch for KAFKA-1438
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/#review44034 --- I get the following error while applying the patch- patching file bin/windows/kafka-run-class.bat Hunk #1 FAILED at 36. Hunk #2 FAILED at 121. 2 out of 2 hunks FAILED -- saving rejects to file bin/windows/kafka-run-class.bat.rej - Neha Narkhede On May 27, 2014, 7:15 p.m., Sriharsha Chintalapani wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/ --- (Updated May 27, 2014, 7:15 p.m.) Review request for kafka. Bugs: KAFKA-1438 https://issues.apache.org/jira/browse/KAFKA-1438 Repository: kafka Description --- KAKFA-1438. Migrate Kafka Client tools. Diffs - bin/kafka-console-consumer.sh e410dde271eea996416f7a67561e45b78f8e8a69 bin/kafka-consumer-perf-test.sh 4ed3ed9fb12edf00a82c06c4e59270a60df43ecd bin/kafka-producer-perf-test.sh b4efc29d76ff8f53a310f812b9ed3de6875465b5 bin/kafka-run-class.sh d2fc8c0e413769bb89ada99790b90ad616044f28 bin/kafka-simple-consumer-perf-test.sh 2d3e3d3be21065a4d446b9f8ed2e9d31c0685456 bin/windows/kafka-console-consumer.bat 94b20a4d1d2d043b4c4280d00b807d1d27089ac1 bin/windows/kafka-run-class.bat f4d2904a3320ae756ed4171f6e99566f1f0cf963 config/tools-log4j.properties 52f07c96019b4083fc78f62cfb0a81080327e847 core/src/main/scala/kafka/consumer/ConsoleConsumer.scala 24c92877cc7df20b82b7a438d39120b1caa3924d core/src/main/scala/kafka/server/OffsetManager.scala 54176283e5f199d9db08e2181e7ddfbb9c801fab perf/config/log4j.properties 542b7391550b2a8bb091a38d08583bea9b3111b6 perf/src/main/scala/kafka/perf/ConsumerPerformance.scala 4dde4687bada1202eaf54be2b06c2156e7726b23 perf/src/main/scala/kafka/perf/PerfConfig.scala a8fc6b9ec81c49c73c35172ddc5a67c6a96c799a perf/src/main/scala/kafka/perf/ProducerPerformance.scala 00fa90bb286900fb33a05f55e3c34c2cbb5620a1 perf/src/main/scala/kafka/perf/SimpleConsumerPerformance.scala c52ada0a30ae49c8d576703859b9d49348f62feb Diff: https://reviews.apache.org/r/21865/diff/ Testing --- Thanks, Sriharsha Chintalapani
Re: Review Request 21865: Patch for KAFKA-1438
On May 27, 2014, 8:40 p.m., Neha Narkhede wrote: I get the following error while applying the patch- patching file bin/windows/kafka-run-class.bat Hunk #1 FAILED at 36. Hunk #2 FAILED at 121. 2 out of 2 hunks FAILED -- saving rejects to file bin/windows/kafka-run-class.bat.rej whats your git version. I am using git 1.8.5.2 on OS X 10.9.2. Probably issue with the older version of git but I'll test it again. Thanks. - Sriharsha --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/#review44034 --- On May 27, 2014, 7:15 p.m., Sriharsha Chintalapani wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/ --- (Updated May 27, 2014, 7:15 p.m.) Review request for kafka. Bugs: KAFKA-1438 https://issues.apache.org/jira/browse/KAFKA-1438 Repository: kafka Description --- KAKFA-1438. Migrate Kafka Client tools. Diffs - bin/kafka-console-consumer.sh e410dde271eea996416f7a67561e45b78f8e8a69 bin/kafka-consumer-perf-test.sh 4ed3ed9fb12edf00a82c06c4e59270a60df43ecd bin/kafka-producer-perf-test.sh b4efc29d76ff8f53a310f812b9ed3de6875465b5 bin/kafka-run-class.sh d2fc8c0e413769bb89ada99790b90ad616044f28 bin/kafka-simple-consumer-perf-test.sh 2d3e3d3be21065a4d446b9f8ed2e9d31c0685456 bin/windows/kafka-console-consumer.bat 94b20a4d1d2d043b4c4280d00b807d1d27089ac1 bin/windows/kafka-run-class.bat f4d2904a3320ae756ed4171f6e99566f1f0cf963 config/tools-log4j.properties 52f07c96019b4083fc78f62cfb0a81080327e847 core/src/main/scala/kafka/consumer/ConsoleConsumer.scala 24c92877cc7df20b82b7a438d39120b1caa3924d core/src/main/scala/kafka/server/OffsetManager.scala 54176283e5f199d9db08e2181e7ddfbb9c801fab perf/config/log4j.properties 542b7391550b2a8bb091a38d08583bea9b3111b6 perf/src/main/scala/kafka/perf/ConsumerPerformance.scala 4dde4687bada1202eaf54be2b06c2156e7726b23 perf/src/main/scala/kafka/perf/PerfConfig.scala a8fc6b9ec81c49c73c35172ddc5a67c6a96c799a perf/src/main/scala/kafka/perf/ProducerPerformance.scala 00fa90bb286900fb33a05f55e3c34c2cbb5620a1 perf/src/main/scala/kafka/perf/SimpleConsumerPerformance.scala c52ada0a30ae49c8d576703859b9d49348f62feb Diff: https://reviews.apache.org/r/21865/diff/ Testing --- Thanks, Sriharsha Chintalapani
Re: Review Request 21865: Patch for KAFKA-1438
On May 27, 2014, 8:40 p.m., Neha Narkhede wrote: I get the following error while applying the patch- patching file bin/windows/kafka-run-class.bat Hunk #1 FAILED at 36. Hunk #2 FAILED at 121. 2 out of 2 hunks FAILED -- saving rejects to file bin/windows/kafka-run-class.bat.rej Sriharsha Chintalapani wrote: whats your git version. I am using git 1.8.5.2 on OS X 10.9.2. Probably issue with the older version of git but I'll test it again. Thanks. This is using patch. patch -p1 -i kafka-1438.patch - Neha --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/#review44034 --- On May 27, 2014, 7:15 p.m., Sriharsha Chintalapani wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/ --- (Updated May 27, 2014, 7:15 p.m.) Review request for kafka. Bugs: KAFKA-1438 https://issues.apache.org/jira/browse/KAFKA-1438 Repository: kafka Description --- KAKFA-1438. Migrate Kafka Client tools. Diffs - bin/kafka-console-consumer.sh e410dde271eea996416f7a67561e45b78f8e8a69 bin/kafka-consumer-perf-test.sh 4ed3ed9fb12edf00a82c06c4e59270a60df43ecd bin/kafka-producer-perf-test.sh b4efc29d76ff8f53a310f812b9ed3de6875465b5 bin/kafka-run-class.sh d2fc8c0e413769bb89ada99790b90ad616044f28 bin/kafka-simple-consumer-perf-test.sh 2d3e3d3be21065a4d446b9f8ed2e9d31c0685456 bin/windows/kafka-console-consumer.bat 94b20a4d1d2d043b4c4280d00b807d1d27089ac1 bin/windows/kafka-run-class.bat f4d2904a3320ae756ed4171f6e99566f1f0cf963 config/tools-log4j.properties 52f07c96019b4083fc78f62cfb0a81080327e847 core/src/main/scala/kafka/consumer/ConsoleConsumer.scala 24c92877cc7df20b82b7a438d39120b1caa3924d core/src/main/scala/kafka/server/OffsetManager.scala 54176283e5f199d9db08e2181e7ddfbb9c801fab perf/config/log4j.properties 542b7391550b2a8bb091a38d08583bea9b3111b6 perf/src/main/scala/kafka/perf/ConsumerPerformance.scala 4dde4687bada1202eaf54be2b06c2156e7726b23 perf/src/main/scala/kafka/perf/PerfConfig.scala a8fc6b9ec81c49c73c35172ddc5a67c6a96c799a perf/src/main/scala/kafka/perf/ProducerPerformance.scala 00fa90bb286900fb33a05f55e3c34c2cbb5620a1 perf/src/main/scala/kafka/perf/SimpleConsumerPerformance.scala c52ada0a30ae49c8d576703859b9d49348f62feb Diff: https://reviews.apache.org/r/21865/diff/ Testing --- Thanks, Sriharsha Chintalapani
Re: Review Request 21865: Patch for KAFKA-1438
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/ --- (Updated May 28, 2014, 12:08 a.m.) Review request for kafka. Bugs: KAFKA-1438 https://issues.apache.org/jira/browse/KAFKA-1438 Repository: kafka Description (updated) --- KAFKA-1438. Migrate Kafka Client tools. Diffs (updated) - bin/kafka-console-consumer.sh e410dde271eea996416f7a67561e45b78f8e8a69 bin/kafka-console-producer.sh cd8ce62fe51e3ad64908146f727f73fc6e35b35e bin/kafka-consumer-perf-test.sh 4ed3ed9fb12edf00a82c06c4e59270a60df43ecd bin/kafka-producer-perf-test.sh b4efc29d76ff8f53a310f812b9ed3de6875465b5 bin/kafka-run-class.sh d2fc8c0e413769bb89ada99790b90ad616044f28 bin/kafka-simple-consumer-perf-test.sh 2d3e3d3be21065a4d446b9f8ed2e9d31c0685456 bin/windows/kafka-console-consumer.bat 94b20a4d1d2d043b4c4280d00b807d1d27089ac1 bin/windows/kafka-console-producer.bat b116e64cc4d8b19e0038f3dd1ff41bc5eb47abab bin/windows/kafka-run-class.bat f4d2904a3320ae756ed4171f6e99566f1f0cf963 core/src/main/scala/kafka/consumer/ConsoleConsumer.scala 1a16c691683dda0c53f316e3c4797ea38e776574 core/src/main/scala/kafka/producer/ConsoleProducer.scala a2af988d99a94a20291d6a2dc9bec73197f1b756 core/src/main/scala/kafka/server/OffsetManager.scala 54176283e5f199d9db08e2181e7ddfbb9c801fab perf/config/log4j.properties 542b7391550b2a8bb091a38d08583bea9b3111b6 perf/src/main/scala/kafka/perf/ConsumerPerformance.scala 4dde4687bada1202eaf54be2b06c2156e7726b23 perf/src/main/scala/kafka/perf/PerfConfig.scala a8fc6b9ec81c49c73c35172ddc5a67c6a96c799a perf/src/main/scala/kafka/perf/ProducerPerformance.scala 00fa90bb286900fb33a05f55e3c34c2cbb5620a1 perf/src/main/scala/kafka/perf/SimpleConsumerPerformance.scala c52ada0a30ae49c8d576703859b9d49348f62feb Diff: https://reviews.apache.org/r/21865/diff/ Testing --- Thanks, Sriharsha Chintalapani
Re: Review Request 21865: Patch for KAFKA-1438
On May 27, 2014, 8:40 p.m., Neha Narkhede wrote: I get the following error while applying the patch- patching file bin/windows/kafka-run-class.bat Hunk #1 FAILED at 36. Hunk #2 FAILED at 121. 2 out of 2 hunks FAILED -- saving rejects to file bin/windows/kafka-run-class.bat.rej Sriharsha Chintalapani wrote: whats your git version. I am using git 1.8.5.2 on OS X 10.9.2. Probably issue with the older version of git but I'll test it again. Thanks. Neha Narkhede wrote: This is using patch. patch -p1 -i kafka-1438.patch I tested the new patch it applies cleanly against the trunk. MessageFormatter in ConsoleConsumer.scala should go into kafka.consumer is it ok to add this in its own file or should it go under existing file. please advise. There are few admin commands kafka.admin.TopicCommand, PreferredReplicaLeaderElectionCommand ReassignPartitionsCommand should they also be moved under tools. - Sriharsha --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/#review44034 --- On May 28, 2014, 12:08 a.m., Sriharsha Chintalapani wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/ --- (Updated May 28, 2014, 12:08 a.m.) Review request for kafka. Bugs: KAFKA-1438 https://issues.apache.org/jira/browse/KAFKA-1438 Repository: kafka Description --- KAFKA-1438. Migrate Kafka Client tools. Diffs - bin/kafka-console-consumer.sh e410dde271eea996416f7a67561e45b78f8e8a69 bin/kafka-console-producer.sh cd8ce62fe51e3ad64908146f727f73fc6e35b35e bin/kafka-consumer-perf-test.sh 4ed3ed9fb12edf00a82c06c4e59270a60df43ecd bin/kafka-producer-perf-test.sh b4efc29d76ff8f53a310f812b9ed3de6875465b5 bin/kafka-run-class.sh d2fc8c0e413769bb89ada99790b90ad616044f28 bin/kafka-simple-consumer-perf-test.sh 2d3e3d3be21065a4d446b9f8ed2e9d31c0685456 bin/windows/kafka-console-consumer.bat 94b20a4d1d2d043b4c4280d00b807d1d27089ac1 bin/windows/kafka-console-producer.bat b116e64cc4d8b19e0038f3dd1ff41bc5eb47abab bin/windows/kafka-run-class.bat f4d2904a3320ae756ed4171f6e99566f1f0cf963 core/src/main/scala/kafka/consumer/ConsoleConsumer.scala 1a16c691683dda0c53f316e3c4797ea38e776574 core/src/main/scala/kafka/producer/ConsoleProducer.scala a2af988d99a94a20291d6a2dc9bec73197f1b756 core/src/main/scala/kafka/server/OffsetManager.scala 54176283e5f199d9db08e2181e7ddfbb9c801fab perf/config/log4j.properties 542b7391550b2a8bb091a38d08583bea9b3111b6 perf/src/main/scala/kafka/perf/ConsumerPerformance.scala 4dde4687bada1202eaf54be2b06c2156e7726b23 perf/src/main/scala/kafka/perf/PerfConfig.scala a8fc6b9ec81c49c73c35172ddc5a67c6a96c799a perf/src/main/scala/kafka/perf/ProducerPerformance.scala 00fa90bb286900fb33a05f55e3c34c2cbb5620a1 perf/src/main/scala/kafka/perf/SimpleConsumerPerformance.scala c52ada0a30ae49c8d576703859b9d49348f62feb Diff: https://reviews.apache.org/r/21865/diff/ Testing --- Thanks, Sriharsha Chintalapani
Re: Review Request 21865: Patch for KAFKA-1438
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/#review44081 --- I get the same error as Neha on kafka-run-class.sh, could you double check if there is anything wrong with that diff? - Guozhang Wang On May 28, 2014, 12:08 a.m., Sriharsha Chintalapani wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21865/ --- (Updated May 28, 2014, 12:08 a.m.) Review request for kafka. Bugs: KAFKA-1438 https://issues.apache.org/jira/browse/KAFKA-1438 Repository: kafka Description --- KAFKA-1438. Migrate Kafka Client tools. Diffs - bin/kafka-console-consumer.sh e410dde271eea996416f7a67561e45b78f8e8a69 bin/kafka-console-producer.sh cd8ce62fe51e3ad64908146f727f73fc6e35b35e bin/kafka-consumer-perf-test.sh 4ed3ed9fb12edf00a82c06c4e59270a60df43ecd bin/kafka-producer-perf-test.sh b4efc29d76ff8f53a310f812b9ed3de6875465b5 bin/kafka-run-class.sh d2fc8c0e413769bb89ada99790b90ad616044f28 bin/kafka-simple-consumer-perf-test.sh 2d3e3d3be21065a4d446b9f8ed2e9d31c0685456 bin/windows/kafka-console-consumer.bat 94b20a4d1d2d043b4c4280d00b807d1d27089ac1 bin/windows/kafka-console-producer.bat b116e64cc4d8b19e0038f3dd1ff41bc5eb47abab bin/windows/kafka-run-class.bat f4d2904a3320ae756ed4171f6e99566f1f0cf963 core/src/main/scala/kafka/consumer/ConsoleConsumer.scala 1a16c691683dda0c53f316e3c4797ea38e776574 core/src/main/scala/kafka/producer/ConsoleProducer.scala a2af988d99a94a20291d6a2dc9bec73197f1b756 core/src/main/scala/kafka/server/OffsetManager.scala 54176283e5f199d9db08e2181e7ddfbb9c801fab perf/config/log4j.properties 542b7391550b2a8bb091a38d08583bea9b3111b6 perf/src/main/scala/kafka/perf/ConsumerPerformance.scala 4dde4687bada1202eaf54be2b06c2156e7726b23 perf/src/main/scala/kafka/perf/PerfConfig.scala a8fc6b9ec81c49c73c35172ddc5a67c6a96c799a perf/src/main/scala/kafka/perf/ProducerPerformance.scala 00fa90bb286900fb33a05f55e3c34c2cbb5620a1 perf/src/main/scala/kafka/perf/SimpleConsumerPerformance.scala c52ada0a30ae49c8d576703859b9d49348f62feb Diff: https://reviews.apache.org/r/21865/diff/ Testing --- Thanks, Sriharsha Chintalapani