So I've been thinking through the architecture of a Twitter-esque system in Couch as a kind of thought exercise to get a better handle on some of the more difficult corners of view generation. What would be the most effective manner of creating Twitter-like status streams?
My initial feeling is to store the followings of a given user as an array in the user's document and also have a view that compiles the followers of a given user. When a user posts a status update, the application would fetch the follower list from that view and simply attach it to the status document. It is then a simply matter of a composite key map of a given status document to all of the users stored within to create a given user's home timeline. Where this breaks down is your @aplusk scenario. Storing a 3.5 million entry array with a document is obviously going to cripple performance (at least I would think it would) as well as take up massive disk space (I estimated around 7MB for a single JSON status with 1MM followers). So if this solution isn't scalable to millions of users, what's an architecture that would be? How do you compose the user's tweet stream such that it can be pulled in an efficient manner? Just trying to start a discussion to help me better understand document-oriented architecture, feel free to ignore me! Michael Bleigh
