The book touches on it and the Propel site has more details about
using doSelectRS to grab resultsets and hydrate objects manually.
Loop through your records in batches of a 1000 or less. - pw
On Dec 17, 1:20 am, James <[EMAIL PROTECTED]> wrote:
> Hi,
>
> It's me again. v1.0.8
>
> The whole reason I started playing with Symfony was for work. I've
> made a personal app to get familiar with Symfony and tonight I decided
> to take it to the work level. We have a database table with 1.4M
> records. We want to index these records in order to do a search on
> them. I was following the askeet example modified for our data set I
> got it to work adding weighs on the words in the company names, but
> hit an error where I ran out of memory. This was not very
> surprising. 1.4M records is a lot of data.
>
> I figured that the best possible way to handle this would be to go
> through each record on at a time. It may take a little while to run,
> but I knew that one record was within the memory limits. Here's my
> code the rest is quite literally out of the askeet example except
> it's not in an object it's simply functions in the batch script.
>
> $countStart=0;
> $countEnd=1400000;
>
> while($countStart != $countEnd){
>
> // batch process here
> $c = new Criteria();
> $c->setLimit(1);
> $c->setOffset($countStart++);
> $company = CompanyPeer::doSelect($c);
>
> $companyName = $company[0]->getCompany();
> $companyId = $company[0]->getId();
>
> $companyNameText = str_repeat(" ".$companyName, 2);
>
> $stemmed = stemPhrase($companyNameText);
>
> $words = array_count_values($stemmed);
>
> updateSearchIndex($companyId,$words);
>
> // Point everything at null.
> $c = null;
> $company = null;
> $companyName = null;
> $companyId = null;
> $componyNameText = null;
> $stemmed = null;
> $words = null;
>
> }
>
> When I run this code It does what it is suppose too up to about 19000
> lines. Then it dumps and says that it ran out of memory again. My
> question is basically how can I work with so many records in Symfony
> without running out of memory, and without increasing how much memory
> php is allowed to use. A similar method worked for me in php, so I am
> wondering what I may be missing.
>
> Thanks,
> James
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"symfony users" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/symfony-users?hl=en
-~----------~----~----~----~------~----~------~--~---
