Evan Huus created KAFKA-1033:
--------------------------------
Summary: Metadata requests do not always return the complete list
of available brokers
Key: KAFKA-1033
URL: https://issues.apache.org/jira/browse/KAFKA-1033
Project: Kafka
Issue Type: Bug
Components: core
Affects Versions: 0.8
Reporter: Evan Huus
I discovered this while writing a Go client (https://github.com/Shopify/sarama)
and it is making one of the issues I'm having rather difficult to solve
(https://github.com/Shopify/sarama/issues/15).
In summary: sending a metadata request with an empty list of topics is supposed
to return a list of *all* metadata in the cluster. However, the list of brokers
is incomplete. I have not been able to pin down precisely which brokers are
missing, but I believe it happens when a broker is not currently the leader for
any partition of any topic.
Among other things this can make it very difficult to provide failover in a
small cluster of only one master and one replica server - clients requesting
metadata sometimes are not told of the replica broker and cannot fail-over to
it when the master goes down.
If it is intentional to only return a subset of brokers (whatever that subset
is), please document somewhere what that subset is, and how clients should
learn of brokers outside that subset.
Thanks,
Evan
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
