On Mon, Apr 15, 2019 at 8:58 PM Michael Sullivan <su...@msully.net> wrote: > > On Mon, Apr 15, 2019 at 4:06 PM Nathaniel Smith <n...@pobox.com> wrote: >> >> On Mon, Apr 15, 2019, 15:27 Michael Sullivan <su...@msully.net> wrote: >>> >>> > The main question is if anyone ever used Py_TRACE_REFS? Does someone >>> > use sys.getobjects() or PYTHONDUMPREFS environment variable? >>> >>> I used sys.getobjects() today to track down a memory leak in the >>> mypyc-compiled version of mypy. >>> >>> We were leaking memory badly but no sign of the leak was showing up in >>> mypy's gc.get_objects() based profiler. Using a debug build and switching >>> to sys.getobjects() showed that we were badly leaking int objects. A quick >>> inspection of the values in question (large and random looking) suggested >>> we were leaking hash values, and that quickly pointed me to >>> https://github.com/mypyc/mypyc/pull/562. >>> >>> I don't have any strong feelings about whether to keep it in the "default" >>> debug build, though. I was using a debug build that I built myself with >>> every debug feature that seemed potentially useful. >> >> >> This is mostly to satisfy my curiosity, so feel free to ignore: did you try >> using address sanitizer or valgrind? >> > I didn't, mostly because I assume that valgrind wouldn't play well with > cpython. (I've never used address sanitizer.) > > I was curious, so I went back and tried it out. > It turned out to not seem to need that much fiddling to get to work. It slows > things down a *lot* and produced 17,000 "loss records", though, so maybe I > don't have it working right. At a glance the records did not shed any light. > > I'd definitely believe that valgrind is up to the task of debugging this, but > my initial take with it shed much less light than my sys.getobjects() > approach. (Though note that my sys.getobjects() approach was slotting it into > an existing python memory profiler we had hacked up, so...)
valgrind on CPython is definitely a bit fiddly – if you need it again you might check out Misc/README.valgrind. Supposedly memory sanitizer is just './configure --with-memory-sanitizer', but I haven't tried it either :-) -n -- Nathaniel J. Smith -- https://vorpus.org _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
