Other good point is: always set a timeout connection when you're getting the
RSS data to avoid your thread get stuck unnecessary. Use cURL (is much more
faster then file_get_contents).

Multithreading in PHP with cURL http://devzone.zend.com/article/3341


Regards,
Igor Escobar
*Software Engineer
*
+ http://blog.igorescobar.com
+ http://www.igorescobar.com
+ @igorescobar <http://www.twitter.com/igorescobar>





On Mon, Sep 12, 2011 at 10:05 AM, Igor Escobar <titiolin...@gmail.com>wrote:

> Use PHP threads. Do the job separately.. in parts... in other words... you
> can't read all them at once.
>
> You can read a little more about php multithreading here:
> http://blog.motane.lu/2009/01/02/multithreading-in-php/
>
> You can use a non-relational database like mongo or couchdb to manage where
> you stop and where you have to look back to the RSS feed as well.
>
> []'s
>
> Regards,
> Igor Escobar
> *Software Engineer
> *
> + http://blog.igorescobar.com
> + http://www.igorescobar.com
> + @igorescobar <http://www.twitter.com/igorescobar>
>
>
>
>
>
>
> On Sat, Sep 10, 2011 at 10:37 PM, Stuart Dallas <stu...@3ft9.com> wrote:
>
>> On 10 Sep 2011, at 09:35, muad shibani wrote:
>>
>> > I want to design an application that reads news from RSS sources.
>> > I have about 1000 RSS feed to collect from.
>> >
>> > I also will use Cron jobs every 15 minutes to collect the data.
>> > the question is: Is there a clever way to collect all those feed items
>> > without exhausting the server
>> > any Ideas
>>
>> I designed a job queuing system a while back when I had a similar problem.
>> You can read about it here: http://stut.net/2009/05/29/php-job-queue/.
>> Set that type of system up and add a job for each feed, set to run every 15
>> minutes. You can then watch the server and tune the number of concurrent job
>> processors so you get the optimum balance between load and speed.
>>
>> -Stuart
>>
>> --
>> Stuart Dallas
>> 3ft9 Ltd
>> http://3ft9.com/
>> --
>> PHP General Mailing List (http://www.php.net/)
>> To unsubscribe, visit: http://www.php.net/unsub.php
>>
>>
>

Reply via email to