taylor great to hear this. how did you achieve ordering without restricting a topic to one broker onw partiton. On Dec 6, 2011 10:01 PM, "Taylor Gautier" <[email protected]> wrote:
> I've already mentioned this before, but I wanted to give a quick shout to > let you guys know that our newest game, Deckadence, is 100% live as of > yesterday. > > Check it out at http://www.tagged.com/deckadence.html > > A little about our use case: > > - Deckadence is a game of buying and selling - or rather trading - > cards. Every user on Tagged owns a card. There are 100M uses on Tagged, > so that means there are 100M cards to trade. > - Kafka enables real-time delivery of events in the game > - An end user browser makes a long-poll event http connection to receive > 1:1 messages and 1:M messages from a specialized http server we built for > this purpose. 1:M messages are delivered from Kafka. > - Because of this design, we can publish a message anywhere inside our > datacenter and send it directly and immediately to any other system that > is > subscribed to Kafka, or to an end-user browser > - Every update event for every card is sent to a unique topic that > represents the users card. > - When a user is browsing any card or list of cards - say a search > result - their browser subscribes to all of the cards on screen. > - The effect of this is that any changes to any card seen on-screen are > seen in real-time by all users of the game > - Our primary producers and consumers are PHP and NodeJS, respectively > > Well, I plan to write up more about this use case in the near future. As > you might have guessed, this is just about as far away from the original > intent of Kafka as you could get - we have PHP that sends messages to > Kafka. Since it's not good to hold a TCP connection open in PHP, we had to > do some trickery here. There was no existing Node client so we had to > write our own. And since there are 100 million users registered on Tagged, > that means we could have in theory 100M topics. Of course in practice we > have far fewer than that. One of the main things we currently have to do > is aggressively clean topics. But basically we have many topics, few > messages (relatively) per topic. And order matters, so we had to deal with > ensuring that we could handle the number of topics we would create, and > ensure ordered delivery and receipt. > > In the future I have big plans for Kafka, another feature is currently in > private test and will be released to the public soon (it uses Kafka in a > more traditional way). And we hope to have many more in 2012... >
