I'd say that relative to reads (and writes) of the news feed, updating friends lists is uncommon. We've sort of settled for an "eventually consistent" model now where we update the recipient lists of the freshest messages immediately and queue the update of older messages using the Tasks API. Really good to know that this path is tried and proven before us. Thanks! And thanks for the jaikuengine link. Strange, I actually knew it was open sourced, but didn't think about exploring it. Yes, you really helped us in our design considerations!
/PEZ On Mar 16, 8:00 pm, "Ikai L (Google)" <[email protected]> wrote: > A couple of thoughts here: > > - is removing of friends a common event? similarly, is adding of > friends a common event? (All relative, relative to "reads" of the news > feed) > - From what I remember, the only way to make heavy reads scale is to > write the data multiple times in peoples' streams. Twitter does this, > from what I remember, using a "eventually consistent" model. This is > why your feed will not update for several minutes when they are under > heavy load. The general consensus, though, is that a relational, > normalized model simply will not work. > - the Jaiku engine is open source for your > study:http://code.google.com/p/jaikuengine. This runs on App Engine > > Hope these help when you're considering a design. > > > > > > On Mon, Mar 15, 2010 at 5:32 AM, PEZ <[email protected]> wrote: > > We first modeled publish-subscribe "fan out" using conventional RDBMS > > thinking. It seemed to work at first, but then, since the IN operator > > works the way it does, we quickly realized we couldn't continue on > > that path. We found Brett Slatkin's presentation from last years > > Google I/O and we have now watched it a few times and also played some > > with thehttp://pubsub-test.appspot.com/example but it isn't clear to > > us how to do it with "dynamic" recipient lists. > > > We want to implement a "News feed" where a user can see messages > > broadcasted by her friends, sorted with newest message first. But the > > feed should reflect changes in her friends list. (If she adds new > > friends, messages from those should be included in the feed, and if > > she removes friends their messages should not be included.) If we use > > the pubsub-test example and attach a recipient list to each message > > this means a lot of manipulation of the recipient lists when users > > connect and disconnect friends. > > > What we need are some hints on how to "think" when modeling this. Does > > this description even make sense? > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]. > > For more options, visit this group > > athttp://groups.google.com/group/google-appengine?hl=en. > > -- > Ikai Lan > Developer Programs Engineer, Google App > Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
