Hi all,
I wonder if there are any tricks that can be used to debug memory
corruption?
I am debugging test_tokenize and test_zipfile64 (from lib_python_tests.py).
If I run test_zipfile64, I sometimes see this error backtrace:
```
testMoreThan64kFilesAppend (test.test_zipfile64.OtherTests) ... RPython
traceback:
File "rpython_jit_metainterp_8.c", line 35597, in CacheEntry_read
File "rpython_rtyper_lltypesystem.c", line 26925, in
ll_dict_getitem__dicttablePtr_objectPtr
memory corruption: bad size for object in the nursery
```
It is definitely related to the RISCV JIT backend which I am working on. I
tried to build a RISCV build with `-O2` and both test_tokenize and
test_zipfile64 passed without problem.
But, I can't further reduce to one of the following case:
1. Bad JIT opcode implementation that results in out-of-bound writes (thus
corrupting the heap data structure).
2. Bad gcmap calculation (thus object is being freed too early or reference
not being relocated properly)
3. Bad malloc* opcode implementation that corrupts the heap.
4. Something else.
For (3), I made two attempts:
a. I tried to skip all "fast paths" and only call the malloc_slowpath
(fixed size, str, unicode, array). But this attempt doesn't help.
b. I tried to build a JIT'ed PyPy targetstandalone.py with `--gc=boehm`,
but, unfortunately, the generated C source code doesn't compile (with the
error message below).
```
pypy_module_cpyext.c: In function
'pypy_g_W_PyCTypeObject__cpyext_attach_pyobj':
pypy_module_cpyext.c:125333:9: warning: implicit declaration of
function 'OP_GC_RAWREFCOUNT_CREATE_LINK_PYOBJ'; did you mean
'OP_GC_RAWREFCOUNT_CREATE_LINK_PYPY'? [-Wimplicit-function-declaration]
125333 | OP_GC_RAWREFCOUNT_CREATE_LINK_PYOBJ(l_v451927,
l_v451928, /* nothing */);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| OP_GC_RAWREFCOUNT_CREATE_LINK_PYPY
pypy_module_cpyext.c:125333:80: error: expected expression before
')' token
125333 | OP_GC_RAWREFCOUNT_CREATE_LINK_PYOBJ(l_v451927,
l_v451928, /* nothing */);
|
^
make: *** [Makefile:762: pypy_module_cpyext.o] Error 1
make: *** Waiting for unfinished jobs....
```
So here comes my question: do we have some way to log the
allocation/marking/relocation/deallocation in the GC?
Or any other suggestions are much appreciated. Thank you.
Regards,
Logan
On Mon, Jan 29, 2024 at 6:51 PM Logan Chien <tzuhsiang.ch...@gmail.com>
wrote:
> Hi CF,
>
> Thank you for your reply.
>
> >> I also ran test_ll_random.py with `--repeat=20000 --random-seed=1234`
> >> and all test are passing.
> >
> > How long does that take, in wall clock time? I think for the other
> > backends we kept it running for a bunch of days after the last crash
> > occurred.
>
> It took only ~6 hrs (wall clock). If it takes a bunch of days on other
> architectures, I guess I must multiply `--repeat` by 40 times or run it on
> real hardware. I'll try it again after I clear other bugs.
>
> Regards,
> Logan
>
> On Mon, Jan 29, 2024 at 12:37 AM CF Bolz-Tereick via pypy-dev <
> pypy-dev@python.org> wrote:
>
>> On 1/29/24 09:27, CF Bolz-Tereick via pypy-dev wrote:
>> > This test looks just wrong, in my opinion. Given that the variable name
>> > is `fff`, I think it was just meant as a check "does it roughly look
>> > like a pointer". So I think somebody just forgot that sys.maxint is not
>> > a power of 2 (and then things failed on win32 and darwin and somebody
>> > fixed it with the extra if). You can change the test to always use
>> > sys.maxint*2+1.
>>
>> I went ahead and just did that change on the main branch.
>>
>> Cheers,
>>
>> CF
>> _______________________________________________
>> pypy-dev mailing list -- pypy-dev@python.org
>> To unsubscribe send an email to pypy-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/pypy-dev.python.org/
>> Member address: tzuhsiang.ch...@gmail.com
>>
>
_______________________________________________
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com
