On Thu, Apr 8, 2010 at 1:29 AM, Juvenn Woo <[email protected]> wrote:
> Hi Micah, > > > Best, > -- > Juvenn Woo ♫ > > > > > On Thu, Apr 8, 2010 at 12:57 AM, Micah <[email protected]> wrote: > >> Hi, >> >> I'm building (yet another) RSS/Atom syndication/syncing service, and >> I'm thinking of basing it on PuSH. My goal is to track external (ie. >> not under my control) RSS/Atom feeds and syndicate them to another >> location. Or, put simpler, I want to replicate an RSS/Atom feed in >> another location. >> >> I started out doing this with Superfeedr, but I ran into a major issue >> with updates. I need the feeds to stay in sync, so if the original >> feed is updated, I need those changes reflected in the syndicated >> display. Superfeedr (afaik) only sends notifications for new posts, >> not updates to existing posts. What about the reference implementation >> running on App Engine? How does it handle updates? >> >> Plus, since I don't control the external feeds, I'd like to use a hub >> that supports polling of feeds that aren't PuSH enabled. It's either >> that or I write my own poller. >> >> So, a summary: >> >> 1) I don't have control of the feeds, and many of them will not be >> PuSH enabled. Those that are not enabled will need to be polled. >> > > That's what exactly Superfeedr does for you. If the topic not PuSH enabled, > then Superfeedr will poll for you. You should not worry about polling > anymore. > > Though, AppEngine's hub will not do that polling for you. > > >> 2) I need to receive updates to the feeds, not just new posts. I'd be >> fine receiving the entire Atom feed every time, as long as someone >> else is doing the polling and just POSTing the entire thing to me. >> > > There're related discussions in this group before. But as far as I know, > the PuSH spec doesn't specify what should be done with the updated entry, > yet. You may like ref > http://pubsubhubbub.googlecode.com/svn/trunk/pubsubhubbub-core-0.3.html#anchor9 > > Sorry, I'd correct myself. It seems PuSH should handle updated entry, to quote from project summary : *The hub **efficiently fetches the published feed*</p/pubsubhubbub/wiki/PublisherEfficiency> > * and multicasts the new/changed content out to all registered > subscribers.* It specified at http://pubsubhubbub.googlecode.com/svn/trunk/pubsubhubbub-core-0.3.html#contentfetch I have no idea whether the AppEngine's hub will fetch you updated entry, sorry! > > >> 3) I don't want to write my own polling service, as that's the real >> heavy lifting here, and if I do that, there's really no place for PuSH >> in my app. >> > > In any cases, you do need PuSH, because it will save you computing > resources. And the PuSH should live with your own polling worker very well, > I guess. > > 4) I can run my own hub, if that would help, but I'd prefer to >> configure one out of the box, as opposed to hacking in the features I >> need. >> > > >> Any recommendations? Am I fundamentally misunderstanding PuSH and what >> it's used for? >> > > No, you're not. That's PuSH should work for, I think. Let's our following > hackers (more professional than me) give you more ideas then. > > >> >> P.S. Sorry if this is a double-post. I tried posting before and it >> never showed up. >> >> >> -- >> To unsubscribe, reply using "remove me" as the subject. >> > >
