I've been spending some quality time with hexedit, vim and a little bit
of python. I can now generate a file which can be used in the unit test.
The problem seems to occur when a '\r' character is right at
readBlockSize boundary, which is 65368 in the current mod_python.util.
I have not yet tested Alexis's fix yet. It's possible that it doesn't
fix the problem but is just masking it since his code uses a
readBlockSize of 65536, which will not be a problem for ugh.pdf.
The attached script will generate a file that can be used for testing. I
haven't tried to integrate it into Nicolas's unit test as I wanted to
share my findings with everyone asap. I'll spend some more time on this
tommorrow.
Happy Bug Hunting,
Jim
Nicolas Lehuen wrote:
Hi guys,
In the pure "if it ain't tested, it ain't fixed" fashion, I've added a
unit test for file upload to the test suite. It uploads a randomly
generated 1 MB file to the server, and check that the MD5 digest
returned by the server is correct. I could not reproduce Alexis' bug
report this way, however. But I then I added a test with the UNIX-HATERS
handbook file ugh.pdf, and bang, here comes the bug.
I've checked in both unit tests into subversion, so that you can play
with them. I'm now going to test Alexis' fix.
#!/usr/bin/env python
import sys
def generate_file(offset=-1, readBlockSize=65368, fname='testfile'):
""" Generate a file which causes the error with file upload
The default offset of -1 should generate a file which will
be corrupted by the file upload.
Test Results:
offset = -2
235a26d377f0b49ba2fd8706f27dc9e5 testfile.-2.bin
235a26d377f0b49ba2fd8706f27dc9e5 testfile.-2.bin.res
offset = -1
** THIS ONE FAILS **
39626fae4bd0812a2522ebe45c23e186 testfile.-1.bin
e1fc1008bfa9a98dde6c9481e3c83c43 testfile.-1.bin.res
offset = 0
8bd673f325f250a90d623defe1fff11d testfile.0.bin
8bd673f325f250a90d623defe1fff11d testfile.0.bin.res
offset = 1
d0f4adf904cdc058f0ac7013baa83998 testfile.1.bin
d0f4adf904cdc058f0ac7013baa83998 testfile.1.bin.res
"""
# readBlockSize is the same as the value from mod_python.util
f = open('%s.%d.src' % (fname, offset), 'w')
f.write('a'*100)
f.write('\r\n')
block_size = readBlockSize + offset
f.write('b'*block_size)
f.write('\rccc')
f.write('d'*100)
f.write('\r\n')
f.close()
if __name__ == '__main__':
generate_file(offset=int(sys.argv[1]))
