Martin Panter added the comment:
After cleaning my build and rebuilding with “./configure --with-pymalloc
--with-pydebug”, I reduced my script to these four lines:
b1 = bytearray(b"abcdefghij") # 10 bytes
del b1[:1]
del b1[:1]
b1 += b"klmnopq" # 7 bytes
Patch bytearray-fix.patch fixes the bug by taking account fact that ob_start is
offset into the allocated memory buffer when checking if a reallocation is
necessary.
Explanation with the unpatched code and my four-line script above:
1. First line allocates 11 bytes of memory (10 for the byte string + 1 NUL
terminator)
2. First “del” reduces the bytearray length to 9 bytes, but actually
reallocates an expanded memory buffer of 16 bytes! (quirky but not the bug)
3. Second “del” reduces the bytearray length to 8 bytes, and increments an
internal ob_start offset without any memory copying or reallocation. (Fine.)
4. Appending step needs to add 7 bytes to the 8-byte array. 7 + 8 + 1 is 16
bytes total required for bytearray and NUL terminator, but since ob_start is
offset from the start of the allocated memory buffer, we overwrite past the end
of the buffer.
Memory debugging output and extra debug printfs of my own:
Resizing to size 10 (current log. offset 0 alloc 0)
=> Major upsize, new alloc = 11
Assigning to linear slice
- Shifting ob_start for negative growth -1
Resizing to size 9 (current log. offset 1 alloc 11)
=> Moderate upsize, new alloc = 16
- Done assigning to linear slice
Assigning to linear slice
- Shifting ob_start for negative growth -1
Resizing to size 8 (current log. offset 1 alloc 16)
=> Minor downsize
- Done assigning to linear slice
Debug memory block at address p=0x7f1af630a0d0: API 'o'
16 bytes originally requested
The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected.
The 8 pad bytes at tail=0x7f1af630a0e0 are not all FORBIDDENBYTE (0xfb):
at tail+0: 0x00 *** OUCH
at tail+1: 0xfb
at tail+2: 0xfb
at tail+3: 0xfb
at tail+4: 0xfb
at tail+5: 0xfb
at tail+6: 0xfb
at tail+7: 0xfb
The block was made by call #32897 to debug malloc/realloc.
Data at p: 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f 70 71
Fatal Python error: bad trailing pad byte
----------
keywords: +patch
Added file: http://bugs.python.org/file39098/bytearray-fix.patch
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue23985>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
