Mark, Most publishers at LinkedIn use a hardware load balancer approach. These are configured to do a TCP healthcheck that monitors if the kafka port on a broker is working. If it is, then requests are forwarded to the broker. Some publishers though are using the software load balancer based on zookeeper. Those applications want to do some key based partitioning of data.
Thanks, Neha On Sat, Nov 5, 2011 at 11:49 AM, Mark <static.void....@gmail.com> wrote: > Sorry but I'm a bit confused now. So at LinkedIn you use a loadbalancer > instead of ZooKeeper or do you use it in conjunction with ZooKeeper? > > Thanks > > On 11/4/11 7:09 PM, Jun Rao wrote: >> >> broker.list is used in the producer property file. One caveat is that the >> broker.list approach doesn't do healthcheck. Which means that if a broker >> goes down, the client could still try to send messages to it. At LinkedIn, >> we rely on a load balancer to do healthcheck for us. The zk-based >> producer, >> on the other hand, does health check. >> >> You can find out more details about our ZK design in our design page in >> the >> website or the paper in >> >> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+papers+and+presentations. >> >> Jun >> >> On Fri, Nov 4, 2011 at 6:52 PM, Mark<static.void....@gmail.com> wrote: >> >>> I just noticed that there is an option to not use Zookeeper and instead >>> one can use a static list of brokers (#9 on >>> http://incubator.apache.org/** >>> >>> kafka/quickstart.html<http://incubator.apache.org/kafka/quickstart.html>). >>> Do i put this list in server.properties? >>> >>> It doesn't seem like you save much either way as you have to either >>> a) list out all the nodes in the zookeeper quorum in >>> zookeeper.properties >>> b) list out static brokers in server.properties. >>> >>> What are the benefits of using ZooKeeper over a static list? Can someone >>> also explain how Kafka uses ZooKeeper? >>> >>> Thanks >>> >>> >
