Welcome and thank you for quickly posting the solution. I've gone ahead and applied the following changeset:
https://github.com/web2py/web2py/commit/1509559e78e858282a7726c6f6cfc553952823e2 to our web2py copy and it looks to have solved the problem. Woot! On Friday, June 21, 2013 4:48:49 AM UTC-7, Massimo Di Pierro wrote: > > Thank you again for reporting this. there is a possible solution in trunk, > proposed by Jonathan. Would you be able to check it? > > On Wednesday, 19 June 2013 17:55:56 UTC-5, Aurelio Tinio wrote: >> >> Hi Massimo, >> We've been able to track down the issue which appeared like a memory >> leak. It seems to be an issue with our use of routes.py. We've been able to >> reproduce the issue with the built-in web server, a smaller application >> (included 'welcome' app) that does not use cache. I've prepared the >> following changeset to show steps to reproduce, with >> routes.example.pyactually being used as routes.py: >> >> >> https://github.com/tinio/web2py/commit/71eb2bee2ca1c7d733bacb7d9da73b1be62f870b >> >> With those changes, as soon as an unknown 404 page is hit (such as >> http://127.0.0.1:8000/welcome/dfsfdf) the amount of memory used by the >> python web2py process increases dramatically, quickly in a few seconds. >> >> >> https://www.evernote.com/shard/s4/sh/7de982fa-c03c-4177-8471-65945c7d6fe3/51b7cbf3248039057b9ae73bb2564434 >> >> https://www.evernote.com/shard/s4/sh/1b211f97-3bc1-430a-b937-3804771fe455/dfab496d33b7eb9d2cc904ff34af5600 >> >> I'm guessing our use of routes.py is incorrect, regardless as promised I >> thought I'd post to get your thoughts. This wasn't immediately obvious to >> us and might be a common pitfall for others that should have safeguards in >> code. Tracking it down, it seems to be an issue with gluon/rewrite.py >> getting caught in an infinite loop. As a quickfix, for our copy of web2py >> we've put in an else break to ensure getting out of the loop. >> >> >> https://github.com/tinio/web2py/commit/c174f4d331d24153b4fc5d2cbb00871db83b62d2 >> >> I still don't fully understand what is being done in the >> try_rewrite_on_error function so confidence in this patch is minimal. >> Again, any thoughts or feedback would be appreciated. >> >> Thanks, >> Aurelio >> >> On Monday, June 10, 2013 7:26:26 PM UTC-7, Massimo Di Pierro wrote: >>> >>> I do not have a good answer but with of all I would try isolate the >>> problem. Can you reproduce it with the built-in web server? Can you >>> reproduce it with a smaller application? Can you reproduce it with am app >>> that does not use cache? etc. If you could post a minimalist code to >>> reproduce it, others could try it too. >>> >>> On Monday, 10 June 2013 19:38:36 UTC-5, Aurelio Tinio wrote: >>>> >>>> Hi Massimo, >>>> I haven't been able to isolate the reason for our memory leak other >>>> than having it be triggered when encountering an unknown page (i.e. 404 >>>> page). The investigation continues and will definitely keep you posted, >>>> especially if the problem is with web2py and not our own application code. >>>> >>>> Curious though.. for the memory leaks that you have found in the past, >>>> what is your process like to track them down? Do you have a preferred >>>> python memory profiler that you use, etc.. ? I'm currently looking into >>>> the >>>> use of Heapy and/or objgraph but figured it wouldn't hurt to ask you >>>> before >>>> I dive deeper. >>>> >>>> Thanks, >>>> Aurelio >>>> ps If this is now way off topic for this thread, happy to repost the >>>> question as a new topic. Please, just let me know. >>>> >>>> On Tuesday, June 4, 2013 8:18:20 PM UTC-7, Massimo Di Pierro wrote: >>>>> >>>>> Let me know if you can isolate the reason for your memory leak. There >>>>> are two known potential causes for leaks. 1) you use cache too much in >>>>> ram >>>>> without clearing the cache; 2) you create instances of objects with a >>>>> __del__ method (this may create circular references which cannot be >>>>> garbage >>>>> collected). None of the web2py classes have a __del__ method but third >>>>> party libraries may. >>>>> >>>>> On Tuesday, 4 June 2013 17:08:50 UTC-5, Aurelio Tinio wrote: >>>>>> >>>>>> Thanks for the prompt response Massimo. It doesn't look like we are >>>>>> using the TAG helper for our application but good to know nonetheless. >>>>>> >>>>>> Cheers. >>>>>> >>>>>> On Tuesday, June 4, 2013 2:42:09 PM UTC-7, Massimo Di Pierro wrote: >>>>>>> >>>>>>> The only memory leak I am aware of is when one use the TAG helper. >>>>>>> It is fixed in trunk and will be foxed in 2.4.8 but it is not fixed in >>>>>>> 2.4.6. I am not aware of other memory leaks. >>>>>>> >>>>>>> On Tuesday, 4 June 2013 16:22:10 UTC-5, Aurelio Tinio wrote: >>>>>>>> >>>>>>>> Hi Massimo, >>>>>>>> If you don't mind, could you elaborate on what these bugfixes >>>>>>>> are? We've just upgraded our system to use v2.4.6 and trying to assess >>>>>>>> if >>>>>>>> it's worth it to do the upgrade to v2.4.7 before our launch. >>>>>>>> Specifically, >>>>>>>> we've noticed a possible memory leak with our deployment and currently >>>>>>>> investigating if this might have been due to our upgrade to v2.4.6 and >>>>>>>> if >>>>>>>> so, if this bug has been fixed in the latest version. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Aurelio >>>>>>>> >>>>>>>> On Friday, May 24, 2013 10:56:11 AM UTC-7, Massimo Di Pierro wrote: >>>>>>>>> >>>>>>>>> I posted web2py 2.4.7. Includes mostly bug fixes. >>>>>>>> >>>>>>>> -- --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
