Thank you for your help. We will investigate.... Any chance you can try if the leak exists with the DAL in web2py stable?
On Friday, 4 November 2016 21:34:52 UTC-5, Sukhmeet Toor wrote: > > Hi web2py-users, > > I have an application that runs on CherryPy (v8.1.2) on Ubuntu (14.04.5 > LTS) and python 2.7.6. I use standalone pyDAL (v16.09) to connect to my > MySQL DB on AWS RDS. I'm seeing a constant memory leak on my servers which > causes the process to use up 100% of RAM in a few hours and eventual die a > painful death (no SWAP enabled; enabling SWAP pushes back certain death a > little). > > I have managed to reproduce the leak with a very small loop that > repeatedly opens and closes the DAL object. I confirmed that memory > continues to grow using htop and will eventually end up using 100% of RAM. > I thought it was an MySQL adapter specific problem but I was able to repro > it with the default sqlite adapter as well. > > from pydal import DAL > > def open_close_dal(): > sql = DAL() > sql.close() > > if __name__ == '__main__': > while True: > open_close_dal() > > > I read some really old threads on this issue that suggested calling > sql._adapter.close() or sql._adapter.close_all_instances() or del sql but it > doesn't seem to make a difference. I've tried tracking down the memory leak > and managed to confirm that it is the guts of the DAL that are leaking. One > iteration of the loop, with pympler.tracker enabled reports these objects > created: > > > types | # objects | total size > ============================================ | =========== | ============ > dict | 672 | 423.38 KB > list | 3525 | 354.35 KB > str | 2851 | 162.49 KB > <class 'collections.OrderedDict | 129 | 135.02 KB > code | 585 | 73.12 KB > <class 'pydal.dialects.MetaDialect | 32 | 28.25 KB > <class 'pydal.parsers.MetaParser | 17 | 15.01 KB > type | 16 | 14.12 KB > <class 'pydal.representers.MetaRepresenter | 16 | 14.12 KB > tuple | 168 | 12.85 KB > <class 'pydal.dialects.sqltype_for | 178 | 11.12 KB > weakref | 89 | 7.65 KB > int | 258 | 6.05 KB > <class 'pydal.representers.for_type | 49 | 3.06 KB > function (wrap) | 25 | 2.93 KB > > > Is this a known issue? I've been pulling out my hair with this for a few days > now and would love any insight into it. > > > Thanks in advance, > > Sukhmeet > > > p.s. Older threads on this topic, which no solution that worked for me: > > https://groups.google.com/forum/#!newtopic/web2py/web2py/hmsupVHdDHo > > http://stackoverflow.com/questions/5539722/standalone-web2py-dal-with-pyramid-requests-threads-and-memory > > > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
