New submission from STINNER Victor <vstin...@redhat.com>:
opcode cache for LOAD_GLOBAL introduced false alarm in memory leak hunting
(python3 -m test -R 3:3 ...).
=> opcache: bpo-26219.
Before the change:
$ git checkout 91234a16367b56ca03ee289f7c03a34d4cfec4c8^
$ make && ./python -m test -R 3:3 test_pprint
...
Tests result: SUCCESS
After the change:
$ git checkout 91234a16367b56ca03ee289f7c03a34d4cfec4c8
$ make && ./python -m test -R 3:3 test_pprint
...
test_pprint leaked [4, 2, 4] memory blocks, sum=10
...
The problem is that at each iteration of regrtest -R 3:3 (6 iterations), a few
more code objects get this opcache allocated.
There are different solutions to fix regrtest -R 3:3.
(*) Always optimize:
diff --git a/Python/ceval.c b/Python/ceval.c
index 411ba3d73c..6cd148efba 100644
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -103,7 +103,7 @@ static long dxp[256];
#endif
/* per opcode cache */
-#define OPCACHE_MIN_RUNS 1024 /* create opcache when code executed this time
*/
+#define OPCACHE_MIN_RUNS 1 /* create opcache when code executed this time */
#define OPCACHE_STATS 0 /* Enable stats */
#if OPCACHE_STATS
$ make && ./python -m test -R 3:3 test_pprint
...
Tests result: SUCCESS
(*) Never optimmize: disable opcache until a better fix can be found
diff --git a/Python/ceval.c b/Python/ceval.c
index 411ba3d73c..3c85df6fea 100644
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -1230,6 +1230,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
f->f_stacktop = NULL; /* remains NULL unless yield suspends frame */
f->f_executing = 1;
+#if 0
if (co->co_opcache_flag < OPCACHE_MIN_RUNS) {
co->co_opcache_flag++;
if (co->co_opcache_flag == OPCACHE_MIN_RUNS) {
@@ -1244,6 +1245,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
#endif
}
}
+#endif
#ifdef LLTRACE
lltrace = _PyDict_GetItemId(f->f_globals, &PyId___ltrace__) != NULL;
$ make && ./python -m test -R 3:3 test_pprint
...
Tests result: SUCCESS
(*) Find a way to explicitly deoptimize all code objects
Modules/gcmodule.c has a clear_freelists() function called by collect() if
generation == NUM_GENERATIONS-1: on when gc.collect() is collected explicitly
for example.
Lib/test/libregrtest/refleak.py also has a dash_R_cleanup() function which
clears many caches.
Problem: currently, code objects are not explicitly tracked (for example, they
are not tracked in a double linked list).
(*) Add way more warmup iterations to regrtest in buildbots.
I dislike this option. A build on a refleak buildbot worker already takes 2 to
3 hours. Adding more warmup would make a build even way more slower.
----------
components: Interpreter Core
messages: 344469
nosy: inada.naoki, vstinner, yselivanov
priority: normal
severity: normal
status: open
title: opcode cache for LOAD_GLOBAL emits false alarm in memory leak hunting
versions: Python 3.8
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue37146>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
