Web servers are simply not designed to have such long single requests The best would be to use shell with short php scripts and some sort of a queue system, which allows you to stop and resume your task, you should also look at parallelizing the task
Being a bit presumptive here your simulation looks like a perfect candidate for using map-reduce HTH Tarique On Tue, Feb 22, 2011 at 5:37 AM, ProFire <[email protected]> wrote: > I use a single http request. > > On Feb 21, 11:32 pm, Maurits van der Schee <[email protected]> wrote: > > Hi, > > > > Do you use a single http request or multiple invoked by a javascript > timer? > > > > Regards, > > > > Maurits > > > > On 02/21/2011 02:49 PM, ProFire wrote: > > > > > > > > > > > > > > > > > Hi fellowcakephpers, > > > > > I've been a long time user ofCakePHPand I've been satisfied since I > > > first tried it. In fact, there's no more turning back for me ever> > since I started it. No framework matchesCakePHPwhen it comes to ease > > > of development. > > > > > However, very recently, I've encountered a problem I can't figure out > > > the source. I'm dealing with Financial Data and being in finance, my > > > application often have to crunch huge sets of data. I've always been > > > very careful with how my application has handled the data as the data > > > involve is huge and I could run into a memory leak if I don't clear > > > those unused variables. > > > > > This year, I was tasked to run a very heavy simulation on the > > > financial data that involves possible 100 million mysql queries in a > > > single run. As such, I'm very prepared to let the simulation run over > > > a period of 1 month. However, within 2 days, the application threw a > > > memory exhausted error. What really puzzled me was I had been very > > > careful not to store any unused data in memory. > > > > > In my algorithm, after I query the data, I store them in a temporary > > > variable. At its final usage, I unset the variable despite knowing > > > that the next iteration the data will be overwritten. That's just to > > > be sure. After each round of simulation, the variables used are stored > > > in the database. After the last $model->save(), I clear every variable > > > used in the simulation, even if the data will be overwritten at the > > > next iteration. > > > > > All other persistent data throughout the iteration are either > > > integers, floats or unchanging arrays. As such, there's no way these > > > persistent data could be the cause of memory leak. > > > > > I've debugged as much as I could to pin point the source of the memory > > > leak in my controllers and models, but without any luck. I ran a > > > smaller simulation and monitor the memory size each iteration, I > > > noticed that the memory either stays the same or gets bigger. > > > > > I still put my faith inCakePHPand I need expert advise on where this > > > memory leak could be. > > -- > Our newest site for the community: CakePHP Video Tutorials > http://tv.cakephp.org > Check out the new CakePHP Questions site http://ask.cakephp.org and help > others with their CakePHP related questions. > > > To unsubscribe from this group, send email to > [email protected] For more options, visit this group > at http://groups.google.com/group/cake-php > -- ============================================================= PHP for E-Biz: http://sanisoft.com ============================================================= -- Our newest site for the community: CakePHP Video Tutorials http://tv.cakephp.org Check out the new CakePHP Questions site http://ask.cakephp.org and help others with their CakePHP related questions. To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/cake-php
