2013/10/6 <mar...@v.loewis.de>: > Quoting Antoine Pitrou <solip...@pitrou.net>: > >> The linecache module is used implicitly by the traceback and warnings >> module, so perhaps quite a bit more than one would imagine :-) >> >> I think limiting the linecache cache size would be good enough. > > So what specific limit would you suggest?
Before using tracemalloc, I didn't even know that lineache has a cache. And Antoine wrote, linecache is used indirectly by many modules. Even if you get a single line, the module does all the whole file content. I was surprised to see that the cache size can be up to 2 MB. (To answer to your question, yes, our application on embedded device does format exceptions and tracebacks, so use linecache.) The re cache is limited. I remember that I saw it once bigger than 700 KB, it was to build a huge range of Unicode characters... but I'm now unable to reproduce the case. I'm quite sure that the cache has an impact on performances. For example, if you format a traceback where most frames come from the same file, you don't want to decode the file from Unicode for each line. I don't want to remove the cache, and I'm unable to configure its size. I suggested a function to clear "all" caches, because *if* you care of memory, you would like to flush all caches (even if you don't know that linecache or re have caches), not anytime, but when your application is idle. I propose a registry because IMO the module should register itself to the registry, the registry should not have to know all modules. (It should not the module to clear its cache ;-)) I worked on a project spending weeks to try to reduce the memory footprint by 1 MB. And it was hard :-) Biggest objects were in the application but it would help to be able to free Python internal caches. Victor _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
