On Sat, Jul 26, 2008 at 5:09 PM, Ralph Meijer <[EMAIL PROTECTED]>wrote: > You couldn't wait until the end of the series did you? ;-) But, this subject is too much fun to wait for! Please do finish your series quickly so that we can all get in the discussion!
> The solution we came up with is using HTML > and/or Atom link elements, on pages/regular > feeds that represent a person's updates, or his > and and his friends, etc. The problem you're solving here is different from and orthogonal to the one I was discussing. Your solution is perfectly reasonable as a means to discover an individual feed and would be particularly useful in finding private feeds. However, it is not well suited to the needs of a service that is trying to provide aggregation, search or discover services over public feeds. Today we have things like Summize and TweetBeep that provide search and alerting services within the limited realm of public content posted to Twitter. But these services rely on special relationships with Twitter.com. That situation is very different from what we have in the more open world of blogging where Google Blogsearch, Technorati, etc. are able to provide services that span across all blogging sites due to the fact that blogs ping and are thus easily discovered by the aggregators. As soon as a new blog goes online, it pings one or more pinging services and immediately has its public postings incorporated into the various services that do blog aggregation, search etc.. This is, I think, a good thing. But, it would be hard to do the same with a solution such as the one you propose (links in HTML, etc.) if only because we can't be sure that all microbloggers would actually have HTML or Atom feeds to carry those links and, even if they did, it can take an awfully long time for an aggregator to find them. The other problem I'm trying to address in my proposal is how one goes about reducing the overhead of handling all the subscriptions that would exist in a large social network where the number of subscribers to any particular feed follows a power law based on the total number of potential subscribers. There are issues of scale here... Imagine that I'm running an XMPP service that has millions of users and you're running a small service that handles several dozen users. Now, imagine that one of your users is very popular and has 1 million of my users as subscribers. If only point-to-point subscriptions are possible, then your service is going to have to provide local storage for 1 million subscriptions and whenever your user publishes data, your service is going to have to either send 1 million copies of the message to my service or at least one copy with 1 million SHIM headers (see XEP-0060). In either case, that's a lot of data! The alternative is for my service to subscribe to your feed of all public data and then do the filtering inside my own servers. (Yes, I should probably report back to you on subscription counts and such... But, that's a different issue.) But, for that to work, I need a way to discover not just a specific feed, but the node which carries all public feeds on your server. I'm not suggesting that a service would advertise something as granular as a single user's feed or that it would advertise private feeds. Rather, services should advertise things like "Feed of all public posts." The advertisment mechanism is, in fact, one that is well understood in the world of general publish/subscribe systems and would, of course, be used to provide discovery for a variety of feeds including those that aren't "social" per se. For instance, a service might advertise that it provides "Weather Feed", "Stock Market Quotes", "Blog Posts", "Ski Resort Conditions", "Movie Show Times", etc... that could be subscribed to, filtered, etc. bob wyman
