Hi - reading through the design doc of Kafka I read that it doesn't support LB:
Currently, there is no built-in load balancing between the producers and the brokers in Kafka; in our own usage we publish from a large number of heterogeneous machines and so it is desirable that the publisher not need any explicit knowledge of the cluster topology. We rely on a hardware load balancer to distribute the producer load across multiple brokers. We will consider adding this in a future release to allow semantic partitioning of messages (i.e. publishing all messages to a particular broker based on some id to ensure an ordered stream of updates within that id). However it says further down: In v0.6, we introduced built-in automatic load balancing between the producers and the brokers in Kafka; Currently, in our own usage we publish from a large number of heterogeneous machines and so it is I assume this is just old documentation however looking through the code (on master) I couldn't find any unit tests for load balancing so I am actually not sure it is supported... Has anyone used it, should it exist? Thanks Florian
