Repository: kafka Updated Branches: refs/heads/trunk d51dd5fc7 -> 4542b027c
KAFKA-3243: Fix Kafka basic ops documentation for Mirror maker, blacklist is not supported for new consumers â¦list is not supported for new consumers Author: Ashish Singh <[email protected]> Reviewers: Grant Henke <[email protected]>, Ewen Cheslack-Postava <[email protected]> Closes #923 from SinghAsDev/KAFKA-3243 Project: http://git-wip-us.apache.org/repos/asf/kafka/repo Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/4542b027 Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/4542b027 Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/4542b027 Branch: refs/heads/trunk Commit: 4542b027c1ac361cc8770383b66abac2129a05e7 Parents: d51dd5f Author: Ashish Singh <[email protected]> Authored: Fri Feb 26 10:37:41 2016 -0800 Committer: Ewen Cheslack-Postava <[email protected]> Committed: Fri Feb 26 10:37:41 2016 -0800 ---------------------------------------------------------------------- .../main/scala/kafka/tools/MirrorMaker.scala | 21 +++++++++++++++----- docs/ops.html | 2 +- 2 files changed, 17 insertions(+), 6 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/kafka/blob/4542b027/core/src/main/scala/kafka/tools/MirrorMaker.scala ---------------------------------------------------------------------- diff --git a/core/src/main/scala/kafka/tools/MirrorMaker.scala b/core/src/main/scala/kafka/tools/MirrorMaker.scala index 95b0aad..4bc38d9 100755 --- a/core/src/main/scala/kafka/tools/MirrorMaker.scala +++ b/core/src/main/scala/kafka/tools/MirrorMaker.scala @@ -173,9 +173,22 @@ object MirrorMaker extends Logging with KafkaMetricsGroup { } CommandLineUtils.checkRequiredArgs(parser, options, consumerConfigOpt, producerConfigOpt) - if (List(whitelistOpt, blacklistOpt).count(options.has) != 1) { - println("Exactly one of whitelist or blacklist is required.") - System.exit(1) + + val useNewConsumer = options.has(useNewConsumerOpt) + if (useNewConsumer) { + if (options.has(blacklistOpt)) { + error("blacklist can not be used when using new consumer in mirror maker. Use whitelist instead.") + System.exit(1) + } + if (!options.has(whitelistOpt)) { + error("whitelist must be specified when using new consumer in mirror maker.") + System.exit(1) + } + } else { + if (List(whitelistOpt, blacklistOpt).count(options.has) != 1) { + error("Exactly one of whitelist or blacklist is required.") + System.exit(1) + } } abortOnSendFailure = options.valueOf(abortOnSendFailureOpt).toBoolean @@ -200,8 +213,6 @@ object MirrorMaker extends Logging with KafkaMetricsGroup { producerProps.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.ByteArraySerializer") producer = new MirrorMakerProducer(producerProps) - val useNewConsumer = options.has(useNewConsumerOpt) - // Create consumers val mirrorMakerConsumers = if (!useNewConsumer) { val customRebalanceListener = { http://git-wip-us.apache.org/repos/asf/kafka/blob/4542b027/docs/ops.html ---------------------------------------------------------------------- diff --git a/docs/ops.html b/docs/ops.html index 33080e2..3cec8d6 100644 --- a/docs/ops.html +++ b/docs/ops.html @@ -118,7 +118,7 @@ Here is an example showing how to mirror a single topic (named <i>my-topic</i>) </pre> Note that we specify the list of topics with the <code>--whitelist</code> option. This option allows any regular expression using <a href="http://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html";>Java-style regular expressions</a>. So you could mirror two topics named <i>A</i> and <i>B</i> using <code>--whitelist 'A|B'</code>. Or you could mirror <i>all</i> topics using <code>--whitelist '*'</code>. Make sure to quote any regular expression to ensure the shell doesn't try to expand it as a file path. For convenience we allow the use of ',' instead of '|' to specify a list of topics. <p> -Sometime it is easier to say what it is that you <i>don't</i> want. Instead of using <code>--whitelist</code> to say what you want to mirror you can use <code>--blacklist</code> to say what to exclude. This also takes a regular expression argument. +Sometimes it is easier to say what it is that you <i>don't</i> want. Instead of using <code>--whitelist</code> to say what you want to mirror you can use <code>--blacklist</code> to say what to exclude. This also takes a regular expression argument. However, <code>--blacklist</code> is not supported when using <code>--new.consumer</code>. <p> Combining mirroring with the configuration <code>auto.create.topics.enable=true</code> makes it possible to have a replica cluster that will automatically create and replicate all data in a source cluster even as new topics are added.
