Can you give a reason why this is? or some links that detail this?
-d
On Dec 18, 2007, at 10:57 AM, pete1978cali wrote:
>
> The book touches on it and the Propel site has more details about
> using doSelectRS to grab resultsets and hydrate objects manually.
> Loop through your records in batches of a 1000 or less. - pw
>
> On Dec 17, 1:20 am, James <[EMAIL PROTECTED]> wrote:
>> Hi,
>>
>> It's me again. v1.0.8
>>
>> The whole reason I started playing with Symfony was for work. I've
>> made a personal app to get familiar with Symfony and tonight I
>> decided
>> to take it to the work level. We have a database table with 1.4M
>> records. We want to index these records in order to do a search on
>> them. I was following the askeet example modified for our data
>> set I
>> got it to work adding weighs on the words in the company names, but
>> hit an error where I ran out of memory. This was not very
>> surprising. 1.4M records is a lot of data.
>>
>> I figured that the best possible way to handle this would be to go
>> through each record on at a time. It may take a little while to run,
>> but I knew that one record was within the memory limits. Here's my
>> code the rest is quite literally out of the askeet example except
>> it's not in an object it's simply functions in the batch script.
>>
>> $countStart=0;
>> $countEnd=1400000;
>>
>> while($countStart != $countEnd){
>>
>> // batch process here
>> $c = new Criteria();
>> $c->setLimit(1);
>> $c->setOffset($countStart++);
>> $company = CompanyPeer::doSelect($c);
>>
>> $companyName = $company[0]->getCompany();
>> $companyId = $company[0]->getId();
>>
>> $companyNameText = str_repeat(" ".$companyName, 2);
>>
>> $stemmed = stemPhrase($companyNameText);
>>
>> $words = array_count_values($stemmed);
>>
>> updateSearchIndex($companyId,$words);
>>
>> // Point everything at null.
>> $c = null;
>> $company = null;
>> $companyName = null;
>> $companyId = null;
>> $componyNameText = null;
>> $stemmed = null;
>> $words = null;
>>
>> }
>>
>> When I run this code It does what it is suppose too up to about
>> 19000
>> lines. Then it dumps and says that it ran out of memory again. My
>> question is basically how can I work with so many records in Symfony
>> without running out of memory, and without increasing how much memory
>> php is allowed to use. A similar method worked for me in php, so I
>> am
>> wondering what I may be missing.
>>
>> Thanks,
>> James
> >
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"symfony users" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/symfony-users?hl=en
-~----------~----~----~----~------~----~------~--~---
