Author: Armin Rigo <[email protected]> Branch: extradoc Changeset: r5762:69d59aa417d0 Date: 2016-12-06 13:18 +0100 http://bitbucket.org/pypy/extradoc/changeset/69d59aa417d0/
Log: Moved to bugs.python.org diff --git a/planning/py3.5/cpython-crashers.rst b/planning/py3.5/cpython-crashers.rst --- a/planning/py3.5/cpython-crashers.rst +++ b/planning/py3.5/cpython-crashers.rst @@ -1,259 +1,11 @@ CPython crashers ================ -This document ways to crash CPython 3.5, or get completely unexpected -and undocumented results, or leak memory, etc. +This used to document ways to crash CPython 3.5, or get completely +unexpected and undocumented results, or leak memory, etc. +It has since been moved to these three main issues: -* _PyGen_Finalize() should not fail with an exception - http://bugs.python.org/issue27811 - -* PyFrameObject.f_gen can be left pointing to a dangling generator - http://bugs.python.org/issue27812 - -* os.scandir() returns an iterable object that should not be used - from multiple threads. Doing so can e.g. cause one thread to - close the dirp while another thread is still using it. This is - likely to crash. Similarly, the test for (!iterator->dirp) at - the start of ScandirIterator_iternext() is only done once even - if the following loop runs two or three times because of "." or - ".." entries. - -* os.scandir() direntry objects should not have stat() called from two - threads concurrently. It will make two stat objects and leak one of - them. - -* _PyGen_yf() checks the opcode at [f_lasti + 1], which is the next - opcode that will run when we resume the generator: either it is the - opcode following the YIELD, or it is exactly YIELD_FROM. It is not - possible at the moment to write Python code that compiles to a YIELD - immediately followed by YIELD_FROM, so by chance the two cases are - correctly distinguished. *However,* the discussion so far assumes - that the generator is not currently running. If it is (which probably - doesn't occur in reasonable Python code but can be constructed - manually), then this checks for example the byte/word that describes - the argument of the currently running opcode. If we're very unlucky - this byte has the value 72, which is YIELD_FROM. Total nonsense and - crashes follow. - -* faulthandler: register(): the signal handler, faulthandler_user(), - changes errno in faulthandler_dump_traceback() but fails to restore it - if chain=False. This can rarely cause random nonsense in the main - program. - -* setting f_lineno didn't evolve when the rest of the bytecodes evolved, - which means it is not safe any more:: - - import sys - - def f(): - try: - raise ValueError # line 5 - except ValueError: - print(42) # line 7 - - def my_trace(*args): - print(args) - if args[1] == 'line': - f = args[0] - if f.f_lineno == 5: - f.f_lineno = 7 - return my_trace - - sys.settrace(my_trace) - f() - sys.settrace(None) - -* I didn't try, but it seems that typeobject.c:mro_internal() is prone - to a refcount crash. It does this:: - - old_mro = type->tp_mro; - ...mro_invoke()... /* might cause reentrance */ - type->tp_mro = new_mro; - ... - Py_XDECREF(old_mro); - - This last XDECREF drops the reference held by the previous value of - ``type->tp_mro`` after we changed it. But ``type->tp_mro`` might have - changed because of mro_invoke(), which calls pure Python code. If it - did change, then old_mro is no longer the old value of - ``type->tp_mro``. The wrong object gets decrefed. - - -Non-segfaulting bugs --------------------- - -* on modern Linux: if the first call in the process to - socketpair() ends in a EINVAL, then cpython will (possibly wrongly) - assume it was caused by SOCK_CLOEXEC and not use SOCK_CLOEXEC at all - in the future - -* fcntl.ioctl(x, y, buf, mutate_flag): mutate_flag is there for the case - of buf being a read-write buffer, which is then mutated in-place. - But if we call with a read-only buffer, mutate_flag is ignored (instead - of rejecting a True value)---ioctl(x, y, "foo", True) will not actually - mutate the string "foo", but the True is completely ignored. (I think - this is a bug introduced during the Argument Clinic refactoring.) - -* re.sub(b'y', bytearray(b'a'), bytearray(b'xyz')) -> b'xaz' - re.sub(b'y', bytearray(b'\\n'), bytearray(b'xyz')) -> internal TypeError - -* if you have a stack of generators where each is in 'yield from' from - the next one, and you call '.next()' on the outermost, then it enters - and leaves all intermediate frames. This is costly but may be - required to get the sys.settrace()/setprofile() hooks called. - However, if you call '.throw()' or '.close()' instead, then it uses a - much more efficient way to go from the outermost to the innermost - frame---as a result, the enter/leave of the intermediate frames is not - invoked. This can confuse coverage tools and profilers. For example, - in a stack ``f1()->f2()->f3()``, vmprof would show f3() as usually - called via f2() from f1() but occasionally called directly from f1(). - -* ceval.c: GET_AITER: calls _PyCoro_GetAwaitableIter(), which might - get an exception from calling the user-defined __await__() or checking - what it returns; such an exception is completely eaten. - -* this is an old issue that was forgotten twice on the - issue tracker: ``class C: __new__=int.__new__`` and ``class C(int): - __new__=object.__new__`` can each be instantiated, even though they - shouldn't. This is because ``__new__`` is completely ignored if it is - set to any built-in function that uses ``tp_new_wrapper`` as its C code - (many of the built-in types' ``__new__`` are like that). - http://bugs.python.org/issue1694663#msg75957, - http://bugs.python.org/issue5322#msg84112. In (at least) CPython 3.5, - a few classes work only thanks to abuse of this bug: for example, - ``io.UnsupportedOperation.__new__(io.UnsupportedOperation)`` doesn't - work, but that was not noticed because ``io.UnsupportedOperation()`` - mistakenly works. - -* this program fails the check for no sys.exc_info(), even though at - the point this assert runs (called from the <== line) we are not in - any except/finally block. This is a generalization of - test_exceptions:test_generator_doesnt_retain_old_exc:: - - import sys - - def g(): - try: - raise ValueError - except ValueError: - yield 1 - assert sys.exc_info() == (None, None, None) - yield 2 - - gen = g() - - try: - raise IndexError - except IndexError: - assert next(gen) is 1 - assert next(gen) is 2 # <== - -* frame.clear() does not clear f_locals, unlike what a test says - (Lib/test/test_frame.py):: - - def test_locals_clear_locals(self): - # Test f_locals before and after clear() (to exercise caching) - f, outer, inner = self.make_frames() - outer.f_locals - inner.f_locals - outer.clear() - inner.clear() - self.assertEqual(outer.f_locals, {}) - self.assertEqual(inner.f_locals, {}) - - This test passes, but the C-level PyFrameObject has got a strong - reference to f_locals, which is only updated (to be empty) if the - Python code tries to read this attribute. In the normal case, - code that calls clear() but doesn't read f_locals afterwards will - still leak everything contained in the C-level f_locals field. This - can be shown by this failing test:: - - import sys - - def g(): - x = 42 - return sys._getframe() - - frame = g() - d = frame.f_locals - frame.clear() - print(d) - assert d == {} # fails! but 'assert d is frame.f_locals' passes, - # which shows that this dict is kept alive by - # 'frame'; and we've seen that it is non-empty - # as long as we don't read frame.f_locals. - -* weak dicts (both kinds) and weak sets have an implementation of - __len__ which doesn't give the "expected" result on PyPy, and in some - cases on CPython too. I'm not sure what is expected and what is not. - Here is an example on CPython 3.5.2+ (using a thread to run the weakref - callbacks only, not to explicitly inspect or modify 'd'):: - - import weakref, _thread - from queue import Queue - - queue = Queue() - def subthread(queue): - while True: - queue.get() - _thread.start_new_thread(subthread, (queue,)) - - class X: - pass - d = weakref.WeakValueDictionary() - while True: - x = X() - d[52] = x - queue.put(x) - del x - while list(d) != []: - pass - assert len(d) == 0 # we've checked that list(d)==[], but this may fail - - On CPython I've seen the assert fail only after editing the function - WeakValueDictionary.__init__.remove() to add ``time.sleep(0.01)`` as - the first line. Otherwise I guess the timings happen to make that test - pass. - -* CPython 3.5.2: this ``nonlocal`` seems not to have a reasonable - effect (note that if we use a different name instead of ``__class__``, - this example correctly complain that there is no binding in the outer - scope of ``Y``):: - - class Y: - class X: - nonlocal __class__ - __class__ = 42 - print(locals()['__class__']) # 42 - print(__class__) # but this is a NameError - -* Follow-up on issue #25388: running ``python x.py`` if x.py contains - the following bytes... - - * ``b"#\xfd\n"`` => we get a SyntaxError: Non-UTF-8 code - * ``b"# coding: utf-8\n#\xfd\n"`` => we get no error! - - -Other issues of "dubious IMHO" status -------------------------------------- - -* argument clinic turns the "bool" specifier into - PyObject_IsTrue(), accepting any argument whatsoever. This can easily - get very confusing for the user, e.g. after messing up the number of - arguments. For example: os.symlink("/path1", "/path2", "/path3") - doesn't fail, it just considers the 3rd argument as some true value. - -* hash({}.values()) works (but hash({}.keys()) correctly gives - TypeError). That's a bit confusing and, as far as I can tell, always - pointless. Also, related: d.keys()==d.keys() but - d.values()!=d.values(). - -* if you write ``from .a import b`` inside the Python prompt, or in - a module not in any package, then you get a SystemError(!) with an - error message that is unlikely to help newcomers. - -* pep 475: unclear why 'os.fchmod(fd)' retries automatically when - it gets EINTR but the otherwise-equivalent 'os.chmod(fd)' does not. - (The documentation says they are fully equivalent, so someone is - wrong.) +http://bugs.python.org/issue28883 +http://bugs.python.org/issue28884 +http://bugs.python.org/issue28885 _______________________________________________ pypy-commit mailing list [email protected] https://mail.python.org/mailman/listinfo/pypy-commit
