On 10 Sep 2011, at 09:35, muad shibani wrote:

> I want to design an application that reads news from RSS sources.
> I have about 1000 RSS feed to collect from.
> I also will use Cron jobs every 15 minutes to collect the data.
> the question is: Is there a clever way to collect all those feed items
> without exhausting the server
> any Ideas

I designed a job queuing system a while back when I had a similar problem. You 
can read about it here: http://stut.net/2009/05/29/php-job-queue/. Set that 
type of system up and add a job for each feed, set to run every 15 minutes. You 
can then watch the server and tune the number of concurrent job processors so 
you get the optimum balance between load and speed.


Stuart Dallas
3ft9 Ltd
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Reply via email to