Hi web2py-users,
I have an application that runs on CherryPy (v8.1.2) on Ubuntu (14.04.5
LTS) and python 2.7.6. I use standalone pyDAL (v16.09) to connect to my
MySQL DB on AWS RDS. I'm seeing a constant memory leak on my servers which
causes the process to use up 100% of RAM in a few hours and eventual die a
painful death (no SWAP enabled; enabling SWAP pushes back certain death a
little).
I have managed to reproduce the leak with a very small loop that repeatedly
opens and closes the DAL object. I confirmed that memory continues to grow
using htop and will eventually end up using 100% of RAM. I thought it was
an MySQL adapter specific problem but I was able to repro it with the
default sqlite adapter as well.
from pydal import DAL
def open_close_dal():
sql = DAL()
sql.close()
if __name__ == '__main__':
while True:
open_close_dal()
I read some really old threads on this issue that suggested calling
sql._adapter.close() or sql._adapter.close_all_instances() or del sql but it
doesn't seem to make a difference. I've tried tracking down the memory leak and
managed to confirm that it is the guts of the DAL that are leaking. One
iteration of the loop, with pympler.tracker enabled reports these objects
created:
types | # objects | total size
============================================ | =========== | ============
dict | 672 | 423.38 KB
list | 3525 | 354.35 KB
str | 2851 | 162.49 KB
<class 'collections.OrderedDict | 129 | 135.02 KB
code | 585 | 73.12 KB
<class 'pydal.dialects.MetaDialect | 32 | 28.25 KB
<class 'pydal.parsers.MetaParser | 17 | 15.01 KB
type | 16 | 14.12 KB
<class 'pydal.representers.MetaRepresenter | 16 | 14.12 KB
tuple | 168 | 12.85 KB
<class 'pydal.dialects.sqltype_for | 178 | 11.12 KB
weakref | 89 | 7.65 KB
int | 258 | 6.05 KB
<class 'pydal.representers.for_type | 49 | 3.06 KB
function (wrap) | 25 | 2.93 KB
Is this a known issue? I've been pulling out my hair with this for a few days
now and would love any insight into it.
Thanks in advance,
Sukhmeet
p.s. Older threads on this topic, which no solution that worked for me:
https://groups.google.com/forum/#!newtopic/web2py/web2py/hmsupVHdDHo
http://stackoverflow.com/questions/5539722/standalone-web2py-dal-with-pyramid-requests-threads-and-memory
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
