Howard, The Kafka model is very different from the Kestrel model, at least as far as consumers are concerned.
Kestrel is like AMQP / RabbitMQ in that it keeps track of consumer state, and you can individually check out messages, and Kestrel will only remove it from the queue when you "commit". Keeping track of this state slows down Kestrel signifiantly though. Kafka does not keep track of consumer state. Each consumer keeps track of an "offset" on its own, and load balancing and offset rollbacks are done completely in the consumer. Kafka doesn't support RabbitMQ-style message checkout and commit, in the server, although you could in theory implement such a consumer yourself. The benefit, though is that Kafka has very little state to track and is wicked fast. In production, Kafka throughput easily trumps Kestrel throughput when you use the checkout/commit features. -Evan On Mon, Oct 8, 2012 at 9:08 AM, howard chen <howac...@gmail.com> wrote: > Hi, > > Someone asked in Quora > ( > http://www.quora.com/Apache-Kafka/What-are-the-most-significant-differences-between-kestrel-and-kafka > ) > and I found this question is particularly interesting, since not many > people has the experience in dealing with these systems. > > My understanding is as following, so pls correct me if I am wrong: > > Same: > - Both are durable and queue's size is limited by disk storage instead of > memory > - There are both very fast > > Difference: > > - No strict ordering kestrel > - Not transactional for kestrel > - No unique delivery guarantee for kestrel > - No zookeeper coordination for kestrel > > So it seems to me that kafka is a more complete solution for message > streaming solution while kestrel is a simplistic implementation of > message queue, am I right? > > Anything to add? > Thanks. > -- -- *Evan Chan* Senior Software Engineer | e...@ooyala.com | (650) 996-4600 www.ooyala.com | blog <http://www.ooyala.com/blog> | @ooyala<http://www.twitter.com/ooyala>
