Yes, memory profiling support is quite poor for Python. I used some code
like the following in the past to track down a nasty memory leak in the ndb
module (https://issuetracker.google.com/u/1/issues/35901184):
@contextlib.contextmanager
def memoryTracker():
if constants.UNITTEST:
yield
return
if not constants.LOCAL:
memory_usage_before = runtime.memory_usage().current()
common_types_before = dict(objgraph.most_common_types(shortnames=False))
yield
gc.collect()
if gc.garbage:
loggin.warn(
'Leaking objects with __del__ methods? (len %d, types %s...)'
% (len(gc.garbage), [g.__class__ for g in gc.garbage[:10]])
)
if not constants.LOCAL:
memory_usage_after = runtime.memory_usage().current()
logging.info(
'Memory usage after: %d (diff: %d)'
% (memory_usage_after, memory_usage_after - memory_usage_before)
)
common_types_after = dict(objgraph.most_common_types(shortnames=False))
common_types_diff = {
name: common_types_after.get(name, 0) -
common_types_before.get(name, 0)
for name in frozenset(common_types_before.keys() +
common_types_after.keys())
}
logging.info(
'Most common types:\n'
+ '\n'.join(
'%s\t%d (diff: %+d)' % (name, common_types_after.get(name, 0),
diff)
for name, diff in sorted(common_types_diff.items(), key=lambda
e: e[1], reverse=True)
)
)
I hereby place the code in the public domain. Feel free to use it in any
way you like. I use it something like:
with memoryTracker():
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit
https://groups.google.com/d/msgid/google-appengine/feabb201-a351-4ae3-8981-cfe4ecfa1363%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
