Evan Huus created KAFKA-2537:
--------------------------------
Summary: Mirrormaker defaults to localhost with no sanity checks
Key: KAFKA-2537
URL: https://issues.apache.org/jira/browse/KAFKA-2537
Project: Kafka
Issue Type: Bug
Components: consumer, replication, zkclient
Affects Versions: 0.8.2.0
Reporter: Evan Huus
Assignee: Neha Narkhede
Short version: Like many other tools, mirror-maker's consumer defaults to using
the localhost zookeeper instance when no specific zookeeper source is
specified. It shouldn't do this. MM should also have a sanity check that the
source and destination clusters are different.
Long version: We run multiple clusters, all using mirrormaker to replicate to
the master cluster. The kafka, zookeeper, and mirrormaker instances all run on
the same nodes in the master cluster since the hardware can more than handle
the load. We were doing some zookeeper maintenance on one of our remote
clusters recently which accidentally caused our configuration manager (chef) to
generate empty zkConnect strings for some mirrormaker instances. These
instances defaulted to localhost and started mirroring from the master cluster
back to itself, an infinite replication loop that caused all sorts of havok.
We were able to recover gracefully and we've added additional safe-guards on
our end, but mirror-maker is at least partially at fault here as well. There is
no reason for it to treat an empty string as anything but an error - especially
not localhost, which is typically the target cluster, not the source.
Additionally, it should be trivial and very useful for mirrormaker to verify it
is not consuming and producing from the same cluster; I can think of no
legitimate use case for this kind of cycle.
If you need any clarification or additional information, please let me know.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
