Hi Massimo, We've been able to track down the issue which appeared like a memory leak. It seems to be an issue with our use of routes.py. We've been able to reproduce the issue with the built-in web server, a smaller application (included 'welcome' app) that does not use cache. I've prepared the following changeset to show steps to reproduce, with routes.example.py actually being used as routes.py:
https://github.com/tinio/web2py/commit/71eb2bee2ca1c7d733bacb7d9da73b1be62f870b With those changes, as soon as an unknown 404 page is hit (such as http://127.0.0.1:8000/welcome/dfsfdf) the amount of memory used by the python web2py process increases dramatically, quickly in a few seconds. https://www.evernote.com/shard/s4/sh/7de982fa-c03c-4177-8471-65945c7d6fe3/51b7cbf3248039057b9ae73bb2564434 https://www.evernote.com/shard/s4/sh/1b211f97-3bc1-430a-b937-3804771fe455/dfab496d33b7eb9d2cc904ff34af5600 I'm guessing our use of routes.py is incorrect, regardless as promised I thought I'd post to get your thoughts. This wasn't immediately obvious to us and might be a common pitfall for others that should have safeguards in code. Tracking it down, it seems to be an issue with gluon/rewrite.py getting caught in an infinite loop. As a quickfix, for our copy of web2py we've put in an else break to ensure getting out of the loop. https://github.com/tinio/web2py/commit/c174f4d331d24153b4fc5d2cbb00871db83b62d2 I still don't fully understand what is being done in the try_rewrite_on_error function so confidence in this patch is minimal. Again, any thoughts or feedback would be appreciated. Thanks, Aurelio On Monday, June 10, 2013 7:26:26 PM UTC-7, Massimo Di Pierro wrote: > > I do not have a good answer but with of all I would try isolate the > problem. Can you reproduce it with the built-in web server? Can you > reproduce it with a smaller application? Can you reproduce it with am app > that does not use cache? etc. If you could post a minimalist code to > reproduce it, others could try it too. > > On Monday, 10 June 2013 19:38:36 UTC-5, Aurelio Tinio wrote: >> >> Hi Massimo, >> I haven't been able to isolate the reason for our memory leak other >> than having it be triggered when encountering an unknown page (i.e. 404 >> page). The investigation continues and will definitely keep you posted, >> especially if the problem is with web2py and not our own application code. >> >> Curious though.. for the memory leaks that you have found in the past, >> what is your process like to track them down? Do you have a preferred >> python memory profiler that you use, etc.. ? I'm currently looking into the >> use of Heapy and/or objgraph but figured it wouldn't hurt to ask you before >> I dive deeper. >> >> Thanks, >> Aurelio >> ps If this is now way off topic for this thread, happy to repost the >> question as a new topic. Please, just let me know. >> >> On Tuesday, June 4, 2013 8:18:20 PM UTC-7, Massimo Di Pierro wrote: >>> >>> Let me know if you can isolate the reason for your memory leak. There >>> are two known potential causes for leaks. 1) you use cache too much in ram >>> without clearing the cache; 2) you create instances of objects with a >>> __del__ method (this may create circular references which cannot be garbage >>> collected). None of the web2py classes have a __del__ method but third >>> party libraries may. >>> >>> On Tuesday, 4 June 2013 17:08:50 UTC-5, Aurelio Tinio wrote: >>>> >>>> Thanks for the prompt response Massimo. It doesn't look like we are >>>> using the TAG helper for our application but good to know nonetheless. >>>> >>>> Cheers. >>>> >>>> On Tuesday, June 4, 2013 2:42:09 PM UTC-7, Massimo Di Pierro wrote: >>>>> >>>>> The only memory leak I am aware of is when one use the TAG helper. It >>>>> is fixed in trunk and will be foxed in 2.4.8 but it is not fixed in >>>>> 2.4.6. >>>>> I am not aware of other memory leaks. >>>>> >>>>> On Tuesday, 4 June 2013 16:22:10 UTC-5, Aurelio Tinio wrote: >>>>>> >>>>>> Hi Massimo, >>>>>> If you don't mind, could you elaborate on what these bugfixes are? >>>>>> We've just upgraded our system to use v2.4.6 and trying to assess if >>>>>> it's >>>>>> worth it to do the upgrade to v2.4.7 before our launch. Specifically, >>>>>> we've >>>>>> noticed a possible memory leak with our deployment and currently >>>>>> investigating if this might have been due to our upgrade to v2.4.6 and >>>>>> if >>>>>> so, if this bug has been fixed in the latest version. >>>>>> >>>>>> Thanks, >>>>>> Aurelio >>>>>> >>>>>> On Friday, May 24, 2013 10:56:11 AM UTC-7, Massimo Di Pierro wrote: >>>>>>> >>>>>>> I posted web2py 2.4.7. Includes mostly bug fixes. >>>>>> >>>>>> -- --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
