This is what I was thinking. Sure there may well be some optimisations that can be done for the process, but I agree that RSS is something that should be checked often.
Which brings me back to my original problem... How do I go about getting this data checked, parsed and dumped into the db in a reliable way that doesn't kill the server? Neil > > Neil: Polling every fifteen minutes is an enormous waste of > > CPU and bandwidth... for both you and the source sites. For > > example, if you're aggregating individual blogs, once every > > 24 hours will cover the vast majority just fine. > > I disagree. RSS was originally built as a solution to provide near > real-time updates on a site. Fullasagoog polls every 15 minutes, not > sure how often MXNA does it. RSS load is a common problem for > website > admins, but I don't ever hear people saying "I need to stop my > visitors > from hitting my site so often", but rather "I need to find a way to > deal > with the load." I know that was a problem for SlashDot when they > first > released RSS for their site. But again, they didn't try to limit the > traffic but rather dealt with the load. If I were using an RSS > reader > (like the squirrel), and it didn't tell me about an update for 24 > hours, > I'd find a different reader. > > > -------------- > > This transmission may contain information that is privileged, > confidential and/or exempt from disclosure under applicable law. If > you are not the intended recipient, you are hereby notified that any > disclosure, copying, distribution, or use of the information contained > herein (including any reliance thereon) is STRICTLY PROHIBITED. If you > received this transmission in error, please immediately contact the > sender and destroy the material in its entirety, whether in electronic > or hard copy format. Thank you. A1. > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Message: http://www.houseoffusion.com/lists.cfm/link=i:4:238198 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4 Donations & Support: http://www.houseoffusion.com/tiny.cfm/54
