Not to be left out, our first Kafka use case at Crowdflower went live on Monday. We've completely replaced a rather large portion of postgres-sweep-based statistics calculations with 2 kafka data channels (handling a major fraction of all the data we touch) and a handful of scala processes. Working well so far, and more uses cases from this data will result over time.
The net result is a rather dramatic drop in write and read load to our master postgres cluster. Our engineering blog will probably talk about it publicly soon, because we're really quite happy with the resulting system; it's simple, performs well, and is quite extensible thanks to the low overhead of adding new consumer groups to existing channels. - dlf On Fri, Nov 18, 2011 at 9:07 AM, Taylor Gautier <tgaut...@tagged.com> wrote: > I just wanted to let you guys know we are live with our first Kafka use case. > > You'll see more from me in the coming weeks about it. > > We released the feature for the public on Monday of this week and have > been slowly rolling it out to our users. > > We will continue to gradually ramp up usage until we get to 100%. > > There is still a lot to do with our implementation and more use cases > are coming soon. > > But so far I am very happy with Kafka it has met all of my > expectations and lived up to all of the claims made by the LinkedIn > team. > > Thank you guys so much for OSS'ing such a great piece of technology. > -- -- Dave Fayram dfay...@gmail.com
