On Mon, Jan 25, 2010 at 5:34 PM, Victor Stinner <victor.stin...@haypocalc.com> wrote: > Hi, > > I'm running regulary my fuzzer (Fusil) on CPython since summer 2008: I tested > Python 2.5, 2.6, 2.7, 3.0, 3.1 and 3.2. I'm only looking for "fatal errors": > Python process killed by a signal, or sometimes fuzzer timeouts. I ignore most > timeout results because most of them are valid function calls reading > from/writing to a file or socket. My goal is to improve Python security: > protect it against malicious data injection and denial of service. I prefer > fuzzing to static code analyze because it finds few false positives and it > directly generate a script reproducing the crash. Fuzzing is just one tool > helping to improve the global security. > > > Bugs found in CPython by Fusil > ============================== > > Modules > ------- > > Fatal errors were only found in modules written in C. Modules: __builtin__ > (5), json (4), io (3), bsddb (3), sqlite3 (3), audioop (2), locale (2), > cProfile (2), Tkinter (2), dl, struct, binascii, testcapi, cPickle, > multibytecodec, ctypes, hotshot, bz2, thread, bisect, weakref, imageop, > multiprocessing. > > __builtin__: Exception, str, unicode, bytearray and long > io: BytesIO, StringIO and FileIO > > It looks like json, bsddb and sqlite3 are young and not enough tested. audioop > and imageop bugs are the most critical because they lead to writing to > uninitialized memory (which might allow to execute arbitrary code). > > This module list gives also a first idea of which modules should be > blacklisted in a sandbox ;-) > > Cause > ----- > > The most common causes are insufficient input validation and invalid/missing > error handling. > > "Insufficient input validation" means that the function is vulnerable to > malicious data injection. "Invalid error handling" means that the function > causes a new error while trying to cleanup data (eg. release memory of an > uninitialized variable). "Missing error handling" means that a function result > is an error but the caller doesn't check the function result. > > I don't have a generic solution to detect these problems. Except for "missing > error handling": gcc has an extension to the C language to indidate that the > result have to be used, __attribute__((warn_unused_result)). The GNU libc uses > it the avoid common bugs. > > > Consequence > ----------- > > The most common consequence is to read from/write to uninitialized memory > (especially reading from a NULL pointer) which lead sometimes to a > segmentation fault (heisenbugs!). The second most common consequence is an > unexpected exception during garbage collection: it displays a Fatal Python > error and quits Python. > > I would suggest to log unexpected exception during garbage collection without > stopping the whole Python process, as done for exceptions in a destructor. > > Details > ------- > > Full list of all bugs found by Fusil with links to the bugtracker and to the > commits: > > http://bitbucket.org/haypo/fusil/wiki/Python > > > Interaction with the Python developers > ====================================== > > I open an issue for each bug found in CPython. I describe how to reproduce it > and try to write a patch. I have learn to always write an unit test, useful to > reproduce the bug, and it makes Python commiters happy :-) > > The reaction depends on the impacted component, the severity of the bug, the > complexity of the code reproducing the bug, and the quality of my bug report > :-) The answer was always quick for core components. But some modules are > maintained by everyone, which means nobody, like imageop, audioop or > cProfile/hotshot. Having a module maitainer, like Guilherme Polo aka gpolo for > Tkiner, does really help! > > It looks like fuzzing bugs are not always appreciated by developers, maybe > because they are always "borderline" cases (not "realist"). > > Sometimes, even if I write a patch, an unit test, explain the problem and the > solution, I don't get any comment. It doesn't motivate me to continue fuzzing > :-/ > > > Play with Fusil at home > ======================= > > If you would like to fuzz Python with Fusil: download the last version of > Fusil and run PYTHON fusil-python as root, where PYTHON is your python > interpreter. Use --success=50 to wait for 50 crashs before stopping, --fast to > speed up the fuzzing but slow down your computer, and --only-c to test only > Python modules written in C. > > http://bitbucket.org/haypo/fusil/wiki/Home > > Fusil is running as the user fusil and group fusil to avoid removing arbitrary > file or killing an arbitrary process, that's why you need to run it as root. > > If Fusil found a crash, you can analyze it while Fusil is running. Go into > python/<crash directory>/ and read stdout and session.log files. Use "sudo > ./replay.py --gdb" command to "replay" the crash in gdb (--valgrind option can > also be useful). > > I'm only working on Linux, but Fusil works on any UNIX/BSD OS. Don't use Fusil > on Windows! It might work on Windows but without any protection for your files > and processes! > > > I hope that my fuzzing tests helped Python project, and may be someone else > would help me to continue these tests ;-) > > Victor Stinner aka haypo
First; thank you - anything that helps find bugs, even edge cases, is good. jesse _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
