I pull about 500K feeds and I use PSHB; mainly to get the updates faster and to reduce the need for frequent pulling. The last time I checked a few months ago, about 60% of the feeds supported PSHB, and it's probably more now.
The basic subscriber implementation was simple. But getting it to work reliably was much harder. The main problem stems from the fact that you can't be sure that you're being notified when a feed is updated. You subscribe to a feed and get the proper confirmation, but then when the feed is updated, it's possible that you don't get any notification (mostly due to the fact there are many movings parts and an error at any point breaks the chain). That forces me to continue to do polling even for feeds that support PSHB (at a slower rate, though). And if I discover that the polling fetches a post that PSHB didn't notify me about, then I mark that PSHB subscription as "failed-to-deliver" and move that feed to the regular polling schedule. There are other little issues as well, here and there, that you discover as you work with many feeds and it adds to the complexity of the implementation. Still, for the feeds for which PSHB works, the benefit is huge. Much faster updates, and a slower polling rate. And that's definitely worth it. If I was starting a new system, though, I probably wouldn't want to manage the subscriptions myself, but offload it to a third party provider such as Julien's offering (which wasn't ready when I was implementing my own). Waleed On Sat, Oct 16, 2010 at 3:17 PM, Bob Wyman <[email protected]> wrote: > Julien, > I think your ideas are good, but I would like to suggest that one thing > that could get more people using PSHB is if we were to encourage people to > think of it as infrastructure that enables whole new kinds of applications > rather than just a improvement on the efficiency of polling for blog and > microblog updates. Frankly, the blog/micro-blog space is pretty densely > occupied and the opportunities to find and exploit under-served niches are > limited or at least hard to discover. Thus, I think that people who are > looking to successfully invest effort in the "next billion dollar idea" are > likely to be the ones that figure out how to use this infrastructure in new > and, as yet, unanticipated ways. > > I am particularly excited by the ability to build real-time event driven > applications that are more focused on "structured data" events rather than > just dissemination of blobs of text. Some examples of structured objects > that could be distributed as Atom entries via PSHB: Offers-To-Sell, > Offers-To-Buy, Pricing data (i.e. products, stocks, etc.), "location" > related data, Weather data, "sensor" data of many kinds, job-openings > (Offers-To-Hire), or resumes (Offers-To-Be-Hired), updates to all kinds of > databases (semantic or otherwise), "Calendar" events, etc... But, I'm sure > I've missed the really interesting thing... > > bob wyman > > > On Sat, Oct 16, 2010 at 4:30 PM, Julien Genestoux < > [email protected]> wrote: > >> Ciao! >> >> A few days ago I was reminded of what initially happend with OpenID. >> Everybody thought it was great, and, after some time, a lot of services >> became providers but not so many of them became consumers. It has changed, >> but I believe we still have a very high producer/consumer ratio. >> >> Today, I am mostly concerned that PubSubHubbub is kind of following the >> same curve. We pretty much have all the big blogging plateforms using >> PubSubHubbub, we have several and services like feedburner make this quite >> ubiquitous for other feeds. >> >> The "consumer" part is also quite impressive, but I would argue that it is >> a little bit less strong. Of course, Google represents a massive subscriber >> but we need more. My question is relatively simple : how could we get more >> subscribers to adopt PubSubHubbub? >> >> I think there are 2 things we should work on : >> >> 1) Find what are the key decision factors for which someone should adopt >> PubSubHubbub (or not!). >> I believe we should ask to those who implementation. I see obvious >> advantages : bandwidth savings, realtime, federation synchronization... >> >> 2) List all the apps who massively poll feeds and maybe try to get in >> touch with all of these guys to see how we could convince them to use >> PubSubHubbub. >> Maybe we should ask those services who host a lot of feeds to check >> their HTTP logs and see what are the UserAgents polling their feeds? I know >> for example that the following companies do poll feeds : LinkedIn, Tumblr, >> Facebook, Netvibes, ... but I'm pretty sure there is a ton of other smaller >> services too. >> >> What do you guys think? >> >> Julien >> >> >> >
