Micah, Obviously, superfeedr wants to implement the detection of updates. It's however _very_ _very_ hard to do. Computing hashed for the entries doesn't work as many feeds havestuff like tracking codes with timestamps. Using <update> tags doesn't work either as they're not widely supported.
I'd be interested to know how the reference hub deals with the updates. If you give me a few weeks, I may have something for you that would work well. Please bear with me. Cheers, On Wed, Apr 7, 2010 at 11:07 PM, Micah <[email protected]> wrote: > Juven, > > Thanks a lot for the reply. I'm glad to know that I'm not crazy trying > to use PuSH. Since Superfeedr doesn't send updates, it's out of the > running. Plus, the AppEngine hub doesn't support polling, so it's out > as well. I believe the AppEngine hub is open source, so perhaps I can > set it up on my own server and enable polling on it somehow. If the > reference implementation supports both polling and sending of updates, > then it would be perfect for my needs. > > Anyone else have any advice? > > On Apr 7, 1:50 pm, Juvenn Woo <[email protected]> wrote: > > On Thu, Apr 8, 2010 at 1:29 AM, Juvenn Woo <[email protected]> wrote: > > > Hi Micah, > > > > > Best, > > > -- > > > Juvenn Woo ♫ > > > > > On Thu, Apr 8, 2010 at 12:57 AM, Micah <[email protected]> > wrote: > > > > >> Hi, > > > > >> I'm building (yet another) RSS/Atom syndication/syncing service, and > > >> I'm thinking of basing it on PuSH. My goal is to track external (ie. > > >> not under my control) RSS/Atom feeds and syndicate them to another > > >> location. Or, put simpler, I want to replicate an RSS/Atom feed in > > >> another location. > > > > >> I started out doing this with Superfeedr, but I ran into a major issue > > >> with updates. I need the feeds to stay in sync, so if the original > > >> feed is updated, I need those changes reflected in the syndicated > > >> display. Superfeedr (afaik) only sends notifications for new posts, > > >> not updates to existing posts. What about the reference implementation > > >> running on App Engine? How does it handle updates? > > > > >> Plus, since I don't control the external feeds, I'd like to use a hub > > >> that supports polling of feeds that aren't PuSH enabled. It's either > > >> that or I write my own poller. > > > > >> So, a summary: > > > > >> 1) I don't have control of the feeds, and many of them will not be > > >> PuSH enabled. Those that are not enabled will need to be polled. > > > > > That's what exactly Superfeedr does for you. If the topic not PuSH > enabled, > > > then Superfeedr will poll for you. You should not worry about polling > > > anymore. > > > > > Though, AppEngine's hub will not do that polling for you. > > > > >> 2) I need to receive updates to the feeds, not just new posts. I'd be > > >> fine receiving the entire Atom feed every time, as long as someone > > >> else is doing the polling and just POSTing the entire thing to me. > > > > > There're related discussions in this group before. But as far as I > know, > > > the PuSH spec doesn't specify what should be done with the updated > entry, > > > yet. You may like ref > > >http://pubsubhubbub.googlecode.com/svn/trunk/pubsubhubbub-core-0.3.ht. > .. > > > > Sorry, I'd correct myself. It seems PuSH should handle updated entry, to > > quote from project summary : > > > > *The hub **efficiently fetches the published > > feed*</p/pubsubhubbub/wiki/PublisherEfficiency> > > > > > * and multicasts the new/changed content out to all registered > > > subscribers.* > > > > It specified athttp:// > pubsubhubbub.googlecode.com/svn/trunk/pubsubhubbub-core-0.3.ht... > > > > I have no idea whether the AppEngine's hub will fetch you updated entry, > > sorry! > > > > > > > > > > > > >> 3) I don't want to write my own polling service, as that's the real > > >> heavy lifting here, and if I do that, there's really no place for PuSH > > >> in my app. > > > > > In any cases, you do need PuSH, because it will save you computing > > > resources. And the PuSH should live with your own polling worker very > well, > > > I guess. > > > > > 4) I can run my own hub, if that would help, but I'd prefer to > > >> configure one out of the box, as opposed to hacking in the features I > > >> need. > > > > >> Any recommendations? Am I fundamentally misunderstanding PuSH and what > > >> it's used for? > > > > > No, you're not. That's PuSH should work for, I think. Let's our > following > > > hackers (more professional than me) give you more ideas then. > > > > >> P.S. Sorry if this is a double-post. I tried posting before and it > > >> never showed up. > > > > >> -- > > >> To unsubscribe, reply using "remove me" as the subject. >
