Use PHP threads. Do the job separately.. in parts... in other words... you
can't read all them at once.

You can read a little more about php multithreading here:

You can use a non-relational database like mongo or couchdb to manage where
you stop and where you have to look back to the RSS feed as well.


Igor Escobar
*Software Engineer
+ @igorescobar <>

On Sat, Sep 10, 2011 at 10:37 PM, Stuart Dallas <> wrote:

> On 10 Sep 2011, at 09:35, muad shibani wrote:
> > I want to design an application that reads news from RSS sources.
> > I have about 1000 RSS feed to collect from.
> >
> > I also will use Cron jobs every 15 minutes to collect the data.
> > the question is: Is there a clever way to collect all those feed items
> > without exhausting the server
> > any Ideas
> I designed a job queuing system a while back when I had a similar problem.
> You can read about it here: Set
> that type of system up and add a job for each feed, set to run every 15
> minutes. You can then watch the server and tune the number of concurrent job
> processors so you get the optimum balance between load and speed.
> -Stuart
> --
> Stuart Dallas
> 3ft9 Ltd
> --
> PHP General Mailing List (
> To unsubscribe, visit:

Reply via email to