[issue37157] shutil: add reflink=False to file copy functions to control clone/CoW copies (use copy_file_range)
Albert Zeyer added the comment: > How is CoW copy supposed to be done by using copy_file_range() exactly? I think copy_file_range() will just always use copy-on-write and/or server-side-copy when available. You cannot even turn that off. -- ___ Python tracker <https://bugs.python.org/issue37157> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37157] shutil: add reflink=False to file copy functions to control clone/CoW copies (use copy_file_range)
Albert Zeyer added the comment: I did some further research (with all details here: https://stackoverflow.com/a/65518879/133374). See vfs_copy_file_range in the Linux kernel. This first tries to call remap_file_range if possible. FICLONE calls ioctl_file_clone. ioctl_file_clone calls vfs_clone_file_range. vfs_clone_file_range calls remap_file_range. I.e. FICLONE == remap_file_range. So using copy_file_range (if available) should be the most generic solution, which includes copy-on-write support, and server-side copy support. -- ___ Python tracker <https://bugs.python.org/issue37157> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37157] shutil: add reflink=False to file copy functions to control clone/CoW copies (use copy_file_range)
Albert Zeyer added the comment: Is FICLONE really needed? Doesn't copy_file_range already supports the same? I posted the same question here: https://stackoverflow.com/questions/65492932/ficlone-vs-ficlonerange-vs-copy-file-range-for-copy-on-write-support -- nosy: +Albert.Zeyer ___ Python tracker <https://bugs.python.org/issue37157> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37159] Use copy_file_range() in shutil.copyfile() (server-side copy)
Albert Zeyer added the comment: According to the man page of copy_file_range (https://man7.org/linux/man-pages/man2/copy_file_range.2.html), copy_file_range also should support copy-on-write: > copy_file_range() gives filesystems an opportunity to implement > "copy acceleration" techniques, such as the use of reflinks > (i.e., two or more inodes that share pointers to the same copy- > on-write disk blocks) or server-side-copy (in the case of NFS). Is this wrong? However, while researching more about FICLONE vs copy_file_range, I found e.g. this: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=24399 Which suggests that there are other problems with copy_file_range? -- nosy: +Albert.Zeyer ___ Python tracker <https://bugs.python.org/issue37159> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39318] NamedTemporaryFile could cause double-close on an fd if _TemporaryFileWrapper throws
Albert Zeyer added the comment: > I think it is worth pointing out that the semantics of > > f = ``open(fd, closefd=True)`` > > are broken (IMHO) because an exception can result in an unreferenced file > object that has taken over reponsibility for closing the fd, but it can > also fail without creating the file object. I thought that if this raises a (normal) exception, it always means that it did not have overtaken the `fd`, i.e. never results in an unreferenced file object which has taken ownership of `fd`. It this is not true right now, you are right that this is problematic. But this should be simple to fix on the CPython side, right? I.e. to make sure whenever some exception is raised here, even if some temporary file object already was constructed, it will not close `fd`. It should be consistent in this behavior, otherwise indeed, the semantics are broken. -- ___ Python tracker <https://bugs.python.org/issue39318> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39318] NamedTemporaryFile could cause double-close on an fd if _TemporaryFileWrapper throws
Albert Zeyer added the comment: If you anyway accept that KeyboardInterrupt can potentially leak, by just using `except Exception`, it would also be solved here. -- ___ Python tracker <https://bugs.python.org/issue39318> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39318] NamedTemporaryFile could cause double-close on an fd if _TemporaryFileWrapper throws
Albert Zeyer added the comment: Why is `except BaseException` better than `except Exception` here? With `except Exception`, you will never run into the problem of possibly closing the fd twice. This is the main important thing which we want to fix here. This is more important than missing maybe to close it at all, or unlink it. -- ___ Python tracker <https://bugs.python.org/issue39318> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39318] NamedTemporaryFile could cause double-close on an fd if _TemporaryFileWrapper throws
Albert Zeyer added the comment: Instead of `except:` and `except BaseException:`, I think better use `except Exception:`. For further discussion and reference, also see the discussion here: https://news.ycombinator.com/item?id=22028581 -- nosy: +Albert.Zeyer ___ Python tracker <https://bugs.python.org/issue39318> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10049] Add a "no-op" (null) context manager to contextlib (Rejected: use contextlib.ExitStack())
Albert Zeyer <alb...@googlemail.com> added the comment: Note that this indeed seems confusing. I just found this thread by search for a null context manager. Because I found that in TensorFlow they introduced _NullContextmanager in their code and I wondered that this is not provided by the Python stdlib. -- nosy: +Albert.Zeyer ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue10049> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24564] shutil.copytree fails when copying NFS to NFS
Albert Zeyer <alb...@googlemail.com> added the comment: I'm also affected by this, with Python 3.6. My home directory is on a ZFS-backed NFS share. See here for details: https://github.com/Linuxbrew/homebrew-core/issues/4799 Basically: Copying setuptools.egg-info to /u/zeyer/.linuxbrew/lib/python3.6/site-packages/setuptools-36.5.0-py3.6.egg-info error: [Errno 5] Input/output error: '/u/zeyer/.linuxbrew/lib/python3.6/site-packages/setuptools-36.5.0-py3.6.egg-info/zip-safe' Note that also by other tools, such as `mv` and `cp`, I get errors about setting `system.nfs4_acl`. But they just seem to ignore that and go on. I think this is the right thing to do here. You can print a warning about that, but then just go on. Maybe esp. just for `system.nfs4_acl`. -- nosy: +Albert.Zeyer versions: +Python 3.6 ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue24564> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31814] subprocess_fork_exec more stable with vfork
Albert Zeyer <alb...@googlemail.com> added the comment: Here is some more background for a case where this occurs: https://stackoverflow.com/questions/46849566/multi-threaded-openblas-and-spawning-subprocesses My proposal here would fix this. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue31814> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31814] subprocess_fork_exec more stable with vfork
Albert Zeyer <alb...@googlemail.com> added the comment: This is a related issue, although with different argumentation: https://bugs.python.org/issue20104 -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue31814> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31814] subprocess_fork_exec more stable with vfork
New submission from Albert Zeyer <alb...@googlemail.com>: subprocess_fork_exec currently calls fork(). I propose to use vfork() or posix_spawn() or syscall(SYS_clone, SIGCHLD, 0) instead if possible and if there is no preexec_fn. The difference would be that fork() will call any atfork handlers (registered via pthread_atfork()), while the suggested calls would not. There are cases where atfork handlers are registered which are not save to be called e.g. in multi-threaded environments. In the case of subprocess_fork_exec without preexec_fn, there is no need to call those atfork handlers, so avoiding this could avoid potential problems. It's maybe acceptable if a pure fork() without exec() doesn't work in this case anymore, but there is no reason that a fork()+exec() should not work in any such cases. This is fixed by my proposed solution. An example case is OpenBlas and OpenMP, which registers an atfork handler which is safe to be called if there are other threads running. See here: https://github.com/tensorflow/tensorflow/issues/13802 https://github.com/xianyi/OpenBLAS/issues/240 https://trac.sagemath.org/ticket/22021 About fork+exec without the atfork handlers, see here for alternatives (like vfork): https://stackoverflow.com/questions/46810597/forkexec-without-atfork-handlers/ -- components: Interpreter Core messages: 304587 nosy: Albert.Zeyer priority: normal severity: normal status: open title: subprocess_fork_exec more stable with vfork type: behavior versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue31814> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30250] StreamIO truncate behavior of current position
New submission from Albert Zeyer:
The doc says that StringIO.truncate should not change the current position.
Consider this code:
try:
import StringIO
except ImportError:
import io as StringIO
buf = StringIO.StringIO()
assert_equal(buf.getvalue(), "")
print("buf: %r" % buf.getvalue())
buf.write("hello")
print("buf: %r" % buf.getvalue())
assert_equal(buf.getvalue(), "hello")
buf.truncate(0)
print("buf: %r" % buf.getvalue())
assert_equal(buf.getvalue(), "")
buf.write("hello")
print("buf: %r" % buf.getvalue())
assert_equal(buf.getvalue(), "\x00\x00\x00\x00\x00hello")
buf.truncate(0)
print("buf: %r" % buf.getvalue())
assert_equal(buf.getvalue(), "")
On Python 3.6, I get the output:
buf: ''
buf: 'hello'
buf: ''
buf: '\x00\x00\x00\x00\x00hello'
On Python 2.7, I get the output:
buf: ''
buf: 'hello'
buf: ''
buf: 'hello'
Thus it seems that Python 2.7 StringIO.truncate does actually resets the
position for this case or there is some other bug in Python 2.7. At least from
the doc, it seems that the Python 3.6 behavior is the expected behavior.
--
components: IO
messages: 292866
nosy: Albert.Zeyer
priority: normal
severity: normal
status: open
title: StreamIO truncate behavior of current position
versions: Python 2.7
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue30250>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23886] faulthandler_user should use _PyThreadState_Current
Albert Zeyer added the comment: Yes exactly. Sorry if I was unclear before. :) I mentioned SIGUSR1/2 because for a segfault, the signal handler will usually be executed in the same thread (although I'm not sure if that is guaranteed), so that was usually not a problem. But I used SIGUSR1 when my Python application is hanging and esp in that case I would like to know the current Python active thread. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue23886> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23886] faulthandler_user should use _PyThreadState_Current
Albert Zeyer added the comment: PyGILState_GetThisThreadState might not be the same Python thread as _PyThreadState_Current, even in the case that both are not NULL. That is because SIGUSR1/2 will get delivered to any running thread. In the output by faulthandler, I want that it marks the current Python thread correctly, and not the current sighandler thread. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue23886> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5710] ctypes should return composite types from callbacks
Albert Zeyer added the comment: Any update here? -- nosy: +Albert.Zeyer versions: +Python 2.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue5710> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1384175] random module - Provider DLL failed to initialize correctly
Albert Zeyer added the comment: Ah thanks, that explains why it failed for me, and why it works after my fix, which was anyway what I intended. I mostly posted my comment here in case someone else hits this, so he has another thing to check/debug. I don't think that there is a bug on Python's side. Maybe a nice thing would be a small check in _PyOS_URandom for os.environ['SystemRoot'] and print a more helpful error. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1384175 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1384175] random module - Provider DLL failed to initialize correctly
Albert Zeyer added the comment:
Note that there are still people who get this error in some strange cases, me
included.
E.g.:
http://stackoverflow.com/questions/27904936/python-exe-file-crashes-while-launching-on-windows-xp/32137554#32137554
This happened at a call to `os.urandom` for me.
This was in a subprocess.
The bug for me was that I called `_subprocess.CreateProcess` with an
`env_mapper = {'foo': 'bar'}`. The fix:
env_mapper = os.environ.copy()
env_mapper.update({'foo': 'bar'})
There is another related question
[here](http://stackoverflow.com/questions/21791005/windows-error-provider-dll-failed-to-initialize-correctly-on-import-of-cgi-mo).
And some discussion on [this GitHub
issue](https://github.com/aws/aws-cli/issues/1226).
All those seem to be related to `crypt32.dll` in a frozen Python app, or via
py2app.
--
nosy: +Albert.Zeyer
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1384175
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23988] keyworded argument count is wrong
New submission from Albert Zeyer: Code: class C(object): def __init__(self, a, b=2, c=3): pass class D(C): def __init__(self, d, **kwargs): super(D, self).__init__(**kwargs) class E(D): def __init__(self, **kwargs): super(E, self).__init__(**kwargs) E(d=42, b=0, c=0) You get the funny message: TypeError: __init__() takes at least 2 arguments (3 given) -- components: Interpreter Core messages: 241335 nosy: Albert.Zeyer priority: normal severity: normal status: open title: keyworded argument count is wrong type: behavior versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23988 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23886] faulthandler_user should use _PyThreadState_Current
New submission from Albert Zeyer: SIGUSR1/2 will get delivered to any running thread. The current thread of the signal doesn't give any useful information. Try to get the current Python thread which holds the GIL instead, or use NULL. I have patched this for the external faulthandler module here: https://github.com/haypo/faulthandler/pull/12 https://github.com/albertz/faulthandler/commit/dc92265 -- components: Library (Lib) messages: 240252 nosy: Albert.Zeyer priority: normal severity: normal status: open title: faulthandler_user should use _PyThreadState_Current type: behavior versions: Python 3.4, Python 3.5, Python 3.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23886 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23291] Documentation about Py_Finalize(): Freeing objects
New submission from Albert Zeyer: The documentation about Py_Finalize() about freeing objects is not exactly clear. Esp., when I have called Py_INCREF somewhere, those objects will always have ob_refcnt 0 unless I call Py_DECREF somewhere, what about these objects? Will they be freed in Py_Finalize() or not? If not, what can I do? I guess calling Py_DECREF after Py_Finalize() is not allowed. I studied the Py_Finalize() code but I'm not exactly sure. It looks like such objects would not get freed (unless some implicit magic is happening somewhere). The comment about the last commented-out PyGC_Collect() call is also insightful and somewhat unsatisfying. Maybe PyObject_Free() still works, although that would not free any further referenced objects, I guess. Also, I'm not exactly sure about the pymalloc arenas. With pymalloc, maybe we could just free all pymalloc arenas and it would free all memory allocated by Python objects? Of course, that would not call any advanced tp_dealloc functions which might be important to be called, but we would never ever call those anyway anymore, or would we (how?)? So, maybe it would be good to do that? -- components: Interpreter Core messages: 234441 nosy: Albert.Zeyer priority: normal severity: normal status: open title: Documentation about Py_Finalize(): Freeing objects versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23291 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20276] ctypes._dlopen should not force RTLD_NOW
New submission from Albert Zeyer: On MacOSX, when you build an ARC-enabled Dylib with backward compatibility for e.g. MacOSX 10.6, some unresolved functions like `_objc_retainAutoreleaseReturnValue` might end up in your Dylib. Some reference about the issue: 1. http://stackoverflow.com/q/21119425/133374. 2. http://osdir.com/ml/python.ctypes/2006-10/msg00029.html 3. https://groups.google.com/forum/#!topic/comp.lang.python/DKmNGwyLl3w Thus, RTLD_NOW is often not an option for MacOSX. This affects mostly `py_dl_open()` from ctypes. But it is also related how you set `dlopenflags` in `PyInterpreterState_New()`. I suggest to make RTLD_LAZY the default on MacOSX (or is there a good reason not to do?). Also, ctypes should have options for both RTLD_NOW and RTLD_LAZY so that both can be used. This is also consistent with the behavior of the [dl module](http://docs.python.org/2/library/dl.html). -- components: ctypes messages: 208226 nosy: Albert.Zeyer priority: normal severity: normal status: open title: ctypes._dlopen should not force RTLD_NOW versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20276 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17294] compile-flag for single-execution to return value instead of printing it
Albert Zeyer added the comment: Thanks a lot for the long and detailed response! I didn't meant to start a header war; I thought that my request was misunderstood and thus the header changes were by mistake. But I guess it is a good suggestion to leave that decision to a core dev. I still thing that this would have been more straight-forward in the first place: for statement in user_input(): if statement: value = exec(compile(statement, 'input', 'single')) if value is not None: print value Because it is more explicit. But because introducing such an incompatible change is bad, I thought it's a good idea to add another compile-mode. Your `ee_compile` seems somewhat inefficient to me because you call `compile` twice and I don't like solutions like this very much (try one thing, then try another thing) as rock-solid solutions. (Of course, neither is `interactive_py_compile`, that one just shows what I want.) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17294 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17294] compile-flag for single-execution to return value instead of printing it
Albert Zeyer added the comment: I don't know that I have an expression and I want it also to work if it is not an expression. Basically I really want the 'single' behavior. (My not-so-uncommon use case: Have an interactive shell where the output on stdout does not make sense. Also I might want to save references to returned values.) displayhook is not an option in any serious bigger project because you don't want to do overwrite that globally. -- resolution: rejected - status: closed - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17294 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17263] crash when tp_dealloc allows other threads
Albert Zeyer added the comment: Btw., where we are at this issue - I have seen many more loops over the threads (via PyThreadState_Next). I have a bad feeling that many of these loops have similar issues. In this case, I am also not sure anymore that it really was a problem. I originally thought that in this loop, it would delete the local-dicts (which contained my Test-object/sqlite connection object). But it does not, it only deallocates a string and the dummy object there. The local-dicts were already been freed at Py_CLEAR(dummies). I still tried to reproduce the crash in the testcase even when the interpreter is not shutting down (like it looks in my musicplayer app) but no success. I also wasn't able yet to get more debugging info about the musicplayer app crash. Note that in the musicplayer app, I have the same workaround now as demonstrated in the testcase and there aren't any crashes anymore (so far - they were seldom anyway). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17263] crash when tp_dealloc allows other threads
Albert Zeyer added the comment: Wouldn't it be better to expose and re-use the HEAD_LOCK and HEAD_UNLOCK macros from pystate.c? The macro-names HEAD_LOCK/HEAD_UNLOCK irritates me a bit. Protecting only the head would not be enough. Any tstate object could be invalidated. But actually, it protects any modification on the list (both in tstate_delete_common and in new_threadstate), as far as I see it. But yes, it would be a good thing to export this locking functionality so other code can use it. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17263] crash when tp_dealloc allows other threads
Albert Zeyer added the comment: Wouldn't it be better to expose and re-use the HEAD_LOCK and HEAD_UNLOCK macros from pystate.c? I don't like holding locks before calling alien code, it's a recipe for deadlocks: for example, if another thread-local object was deallocated as part of the deallocation chain, we would call back into local_clear(), and deadlock. Ah, yes. Right now, the head-lock is acquired while the GIL is held. So while the head-lock is held, we must not unlock the GIL. So this wouldn't work. Btw., I think it also does happen already. While playing around with this test case, I sometimes encountered a deadlock at quit. I was thinking that it was the result of some badly written memory. But I just saw this code (PyInterpreterState_Clear): HEAD_LOCK(); for (p = interp-tstate_head; p != NULL; p = p-next) PyThreadState_Clear(p); HEAD_UNLOCK(); So, if something inside PyThreadState_Clear unlocks the GIL and some other thread acquires the GIL and then tries to HEAD_LOCK (for example, at thread exit), you have a classic deadlock. A solution would be: Only acquire the head-mutex while the GIL is not held. Then, after you held the head-mutex, also acquire the GIL. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17263] crash when tp_dealloc allows other threads
Albert Zeyer added the comment: Btw., this turns out to be at least 4 kind of separate bugs: 1. The crash from the testcase - when the interpreter shuts down. 2. Maybe the crash from my musicplayer app - if that is a different one. But very related to the first one. 3. Many loops over the thread states could have code inside which might release the GIL. All these loops can crash because the thread state could be invalidated in the meanwhile. 4. Possible deadlock with HEAD_LOCK usage. Should we make separate issue reports for each? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17294] compile-flag for single-execution to return value instead of printing it
New submission from Albert Zeyer: `compile(s, interactive, single)` would generate a code object which prints the value of the evaluated string if that is an expression. This is what you would normally want in a REPL. Instead of printing the value, it might make more sense to return it and to leave it to the developer - there are many cases where it shouldn't end up on stdout but somewhere else. There could be an additional compile-flag which would make a code-object returning the value instead of printing it. Note that I have come up with a workaround: def interactive_py_compile(source, filename=interactive): c = compile(source, filename, single) # we expect this at the end: # PRINT_EXPR # LOAD_CONST # RETURN_VALUE import dis if ord(c.co_code[-5]) != dis.opmap[PRINT_EXPR]: return c assert ord(c.co_code[-4]) == dis.opmap[LOAD_CONST] assert ord(c.co_code[-1]) == dis.opmap[RETURN_VALUE] code = c.co_code[:-5] code += chr(dis.opmap[RETURN_VALUE]) CodeArgs = [ argcount, nlocals, stacksize, flags, code, consts, names, varnames, filename, name, firstlineno, lnotab, freevars, cellvars] c_dict = dict([(arg, getattr(c, co_ + arg)) for arg in CodeArgs]) c_dict[code] = code import types c = types.CodeType(*[c_dict[arg] for arg in CodeArgs]) return c My related StackOverflow question: http://stackoverflow.com/questions/15059372/python-use-of-eval-in-interactive-terminal-how-to-get-return-value-what-compi -- components: Interpreter Core messages: 182934 nosy: Albert.Zeyer priority: normal severity: normal status: open title: compile-flag for single-execution to return value instead of printing it type: enhancement versions: Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4, Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17294 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17263] crash when tp_dealloc allows other threads
Albert Zeyer added the comment:
The symbols are there because it is a library which exports all the symbols.
Other debugging information are not there and I don't know any place where I
can get them.
It currently cannot work on Linux in the same way because the GUI is Cocoa only
right now. I'm trying to get it to run with another Python on Mac, though.
Note that in threadmodule.c, in local_clear, we are iterating through all
threads:
/* Remove all strong references to dummies from the thread states */
if (self-key
(tstate = PyThreadState_Get())
tstate-interp) {
for(tstate = PyInterpreterState_ThreadHead(tstate-interp);
tstate;
tstate = PyThreadState_Next(tstate))
if (tstate-dict
PyDict_GetItem(tstate-dict, self-key))
PyDict_DelItem(tstate-dict, self-key);
}
In PyDict_DelItem, if the GIL is released and meanwhile, the list of
threadstates is altered, is that a problem for this loop? So maybe tstate
becomes invalid there.
I also noticed this part in another backtrace of the same crash:
Thread 2:
0 libsystem_kernel.dylib 0x7fff8a54e0fa __psynch_cvwait + 10
1 libsystem_c.dylib 0x7fff85daaf89 _pthread_cond_wait + 869
2 org.python.python 0x00010006f54e PyThread_acquire_lock +
96
3 org.python.python 0x00010001d8e3 PyEval_RestoreThread + 61
4 org.python.python 0x000100053351 PyGILState_Ensure + 93
5 _objc.so0x000103b89b6e 0x103b8 + 39790
6 libobjc.A.dylib 0x7fff880c6230 (anonymous
namespace)::AutoreleasePoolPage::pop(void*) + 464
7 libobjc.A.dylib 0x7fff880c85a2 (anonymous
namespace)::AutoreleasePoolPage::tls_dealloc(void*) + 42
8 libsystem_c.dylib 0x7fff85dad4fe _pthread_tsd_cleanup +
240
9 libsystem_c.dylib 0x7fff85da69a2 _pthread_exit + 146
10 libsystem_c.dylib 0x7fff85da674d _pthread_start + 338
11 libsystem_c.dylib 0x7fff85d93181 thread_start + 13
This seems to be a non-Python thread, so PyGILState_Ensure would have created a
new threadstate and this would have altered the list.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17263
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17263] crash when tp_dealloc allows other threads
Albert Zeyer added the comment: Note that in my original application where I encountered this (with sqlite), the backtrace looks slightly different. It is at shutdown, but not at interpreter shutdown - the main thread is still running. https://github.com/albertz/music-player/issues/23 I was trying to reproduce it in a similar way with this test case but in the test case, so far I could only reproduce the crash when it does the interpreter shutdown. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17263] crash when tp_dealloc allows other threads
Albert Zeyer added the comment: Here is one. Others are in the issue report on GitHub. In Thread 5, the PyObject_SetAttr is where some attribute containing a threading.local object is set to None. This threading.local object had a reference to a sqlite connection object (in some TLS contextes). This should also be the actual crashing thread. I use faulthandler which makes it look like Thread 0 crashed in the crash reporter. I had this crash about 5% of the time - but totally unpredictable. But it was always happening in exactly that line where the attribute was set to None. Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 libsystem_kernel.dylib 0x7fff8a54e0fa __psynch_cvwait + 10 1 libsystem_c.dylib 0x7fff85daaf89 _pthread_cond_wait + 869 2 org.python.python 0x00010006f54e PyThread_acquire_lock + 96 3 org.python.python 0x00010001d8e3 PyEval_RestoreThread + 61 4 org.python.python 0x000100075bf3 0x19000 + 445427 5 org.python.python 0x000100020041 PyEval_EvalFrameEx + 7548 6 org.python.python 0x00010001e281 PyEval_EvalCodeEx + 1956 7 org.python.python 0x000100024661 0x19000 + 112225 8 org.python.python 0x0001000200d2 PyEval_EvalFrameEx + 7693 9 org.python.python 0x00010001e281 PyEval_EvalCodeEx + 1956 10 org.python.python 0x000100024661 0x19000 + 112225 11 org.python.python 0x0001000200d2 PyEval_EvalFrameEx + 7693 12 org.python.python 0x00010001e281 PyEval_EvalCodeEx + 1956 13 org.python.python 0x00010005df78 0x19000 + 348024 14 org.python.python 0x00010001caba PyObject_Call + 97 15 _objc.so0x000104615898 0x10460 + 88216 16 libffi.dylib0x7fff8236e8a6 ffi_closure_unix64_inner + 508 17 libffi.dylib0x7fff8236df66 ffi_closure_unix64 + 70 18 com.apple.AppKit0x7fff84f63f3f -[NSApplication _docController:shouldTerminate:] + 75 19 com.apple.AppKit0x7fff84f63e4e __91-[NSDocumentController(NSInternal) _closeAllDocumentsWithDelegate:shouldTerminateSelector:]_block_invoke_0 + 159 20 com.apple.AppKit0x7fff84f63cea -[NSDocumentController(NSInternal) _closeAllDocumentsWithDelegate:shouldTerminateSelector:] + 1557 21 com.apple.AppKit0x7fff84f636ae -[NSDocumentController(NSInternal) __closeAllDocumentsWithDelegate:shouldTerminateSelector:] + 265 22 com.apple.AppKit0x7fff84f6357f -[NSApplication _shouldTerminate] + 772 23 com.apple.AppKit0x7fff84f9134f -[NSApplication(NSAppleEventHandling) _handleAEQuit] + 403 24 com.apple.AppKit0x7fff84d40261 -[NSApplication(NSAppleEventHandling) _handleCoreEvent:withReplyEvent:] + 660 25 com.apple.Foundation0x7fff867e112b -[NSAppleEventManager dispatchRawAppleEvent:withRawReply:handlerRefCon:] + 308 26 com.apple.Foundation0x7fff867e0f8d _NSAppleEventManagerGenericHandler + 106 27 com.apple.AE0x7fff832eeb48 aeDispatchAppleEvent(AEDesc const*, AEDesc*, unsigned int, unsigned char*) + 307 28 com.apple.AE0x7fff832ee9a9 dispatchEventAndSendReply(AEDesc const*, AEDesc*) + 37 29 com.apple.AE0x7fff832ee869 aeProcessAppleEvent + 318 30 com.apple.HIToolbox 0x7fff8e19f8e9 AEProcessAppleEvent + 100 31 com.apple.AppKit0x7fff84d3c916 _DPSNextEvent + 1456 32 com.apple.AppKit0x7fff84d3bed2 -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 128 33 com.apple.AppKit0x7fff84d33283 -[NSApplication run] + 517 34 libffi.dylib0x7fff8236dde4 ffi_call_unix64 + 76 35 libffi.dylib0x7fff8236e619 ffi_call + 853 36 _objc.so0x00010461a663 PyObjCFFI_Caller + 1980 37 _objc.so0x00010462f43e 0x10460 + 193598 38 org.python.python 0x00010001caba PyObject_Call + 97 39 org.python.python 0x000100020225 PyEval_EvalFrameEx + 8032 40 org.python.python 0x0001000245eb 0x19000 + 112107 41 org.python.python 0x0001000200d2 PyEval_EvalFrameEx + 7693 42 org.python.python 0x00010001e281 PyEval_EvalCodeEx + 1956 43 org.python.python 0x00010001dad7 PyEval_EvalCode + 54 44 org.python.python 0x000100054933 0x19000 + 309555 45 org.python.python 0x0001000549ff PyRun_FileExFlags + 165 46 org.python.python 0x0001000543e9 PyRun_SimpleFileExFlags + 410 47 albertzeyer.MusicPlayer
[issue17263] crash when tp_dealloc allows other threads
Albert Zeyer added the comment: Sadly, that is quite complicated or almost impossible. It needs the MacOSX system Python and that one lacks debugging information. I just tried with the CPython vom hg-2.7. But it seems the official Python doesn't have objc bindings (and I also need Cocoa bindings) so I can't easily run this right now (and another GUI is not yet implemented). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17263] crash when tp_dealloc allows other threads
Albert Zeyer added the comment: The latest 2.7 hg still crashes. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17263] crash when tp_dealloc allows other threads
Albert Zeyer added the comment: The backtrace: Thread 0:: Dispatch queue: com.apple.main-thread 0 libsystem_kernel.dylib 0x7fff8a54e386 __semwait_signal + 10 1 libsystem_c.dylib 0x7fff85e30800 nanosleep + 163 2 libsystem_c.dylib 0x7fff85e30717 usleep + 54 3 testcrash_python_threadlocal.so 0x0001002ddd40 test_dealloc + 48 4 python.exe 0x0001000400a9 dict_dealloc + 153 (dictobject.c:1010) 5 python.exe 0x0001000432d3 PyDict_DelItem + 259 (dictobject.c:855) 6 python.exe 0x0001000d7f27 _localdummy_destroyed + 71 (threadmodule.c:585) 7 python.exe 0x00016c61 PyObject_Call + 97 (abstract.c:2529) 8 python.exe 0x00016e42 PyObject_CallFunctionObjArgs + 370 (abstract.c:2761) 9 python.exe 0x00010006b2e6 PyObject_ClearWeakRefs + 534 (weakrefobject.c:892) 10 python.exe 0x0001000d746b localdummy_dealloc + 27 (threadmodule.c:231) 11 python.exe 0x0001000400a9 dict_dealloc + 153 (dictobject.c:1010) 12 python.exe 0x0001000c003b PyThreadState_Clear + 139 (pystate.c:240) 13 python.exe 0x0001000c02c8 PyInterpreterState_Clear + 56 (pystate.c:104) 14 python.exe 0x0001000c1c68 Py_Finalize + 344 (pythonrun.c:504) 15 python.exe 0x0001000d5891 Py_Main + 3041 (main.c:665) 16 python.exe 0x00010a74 start + 52 Thread 1: 0 libsystem_kernel.dylib 0x7fff8a54e386 __semwait_signal + 10 1 libsystem_c.dylib 0x7fff85e30800 nanosleep + 163 2 libsystem_c.dylib 0x7fff85e30717 usleep + 54 3 testcrash_python_threadlocal.so 0x0001002ddd40 test_dealloc + 48 4 python.exe 0x0001000400a9 dict_dealloc + 153 (dictobject.c:1010) 5 python.exe 0x0001000432d3 PyDict_DelItem + 259 (dictobject.c:855) 6 python.exe 0x0001000d7f27 _localdummy_destroyed + 71 (threadmodule.c:585) 7 python.exe 0x00016c61 PyObject_Call + 97 (abstract.c:2529) 8 python.exe 0x00016e42 PyObject_CallFunctionObjArgs + 370 (abstract.c:2761) 9 python.exe 0x00010006b2e6 PyObject_ClearWeakRefs + 534 (weakrefobject.c:892) 10 python.exe 0x0001000d746b localdummy_dealloc + 27 (threadmodule.c:231) 11 python.exe 0x0001000400a9 dict_dealloc + 153 (dictobject.c:1010) 12 python.exe 0x0001000c003b PyThreadState_Clear + 139 (pystate.c:240) 13 python.exe 0x0001000d7ec4 t_bootstrap + 372 (threadmodule.c:643) 14 libsystem_c.dylib 0x7fff85da6742 _pthread_start + 327 15 libsystem_c.dylib 0x7fff85d93181 thread_start + 13 Thread 2: 0 libsystem_kernel.dylib 0x7fff8a54e322 __select + 10 1 time.so 0x0001002fb01b time_sleep + 139 (timemodule.c:948) 2 python.exe 0x00010009fcfb PyEval_EvalFrameEx + 18011 (ceval.c:4021) 3 python.exe 0x0001000a30f3 fast_function + 179 (ceval.c:4107) 4 python.exe 0x00010009fdad PyEval_EvalFrameEx + 18189 (ceval.c:4042) 5 python.exe 0x0001000a2fb7 PyEval_EvalCodeEx + 2103 (ceval.c:3253) 6 python.exe 0x00010002f8cb function_call + 347 (funcobject.c:526) 7 python.exe 0x00016c61 PyObject_Call + 97 (abstract.c:2529) 8 python.exe 0x0001000a066a PyEval_EvalFrameEx + 20426 (ceval.c:4334) 9 python.exe 0x0001000a30f3 fast_function + 179 (ceval.c:4107) 10 python.exe 0x00010009fdad PyEval_EvalFrameEx + 18189 (ceval.c:4042) 11 python.exe 0x0001000a30f3 fast_function + 179 (ceval.c:4107) 12 python.exe 0x00010009fdad PyEval_EvalFrameEx + 18189 (ceval.c:4042) 13 python.exe 0x0001000a2fb7 PyEval_EvalCodeEx + 2103 (ceval.c:3253) 14 python.exe 0x00010002f8cb function_call + 347 (funcobject.c:526) 15 python.exe 0x00016c61 PyObject_Call + 97 (abstract.c:2529) 16 python.exe 0x000100018b07 instancemethod_call + 439 (classobject.c:2603) 17 python.exe 0x00016c61 PyObject_Call + 97 (abstract.c:2529) 18 python.exe
[issue17263] crash when tp_dealloc allows other threads
New submission from Albert Zeyer: If you have some Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS in some tp_dealloc and you use such objects in thread local storage, you might get crashes, depending on which thread at what time is trying to cleanup such object. I haven't fully figured out the details but I have a somewhat reduced testcase. Note that I encountered this in practice because the sqlite connection object does that (while it disconnects, the GIL is released). This is the C code with some dummy type which has a tp_dealloc which just sleeps for some seconds while the GIL is released: https://github.com/albertz/playground/blob/master/testcrash_python_threadlocal.c This is the Python code: https://github.com/albertz/playground/blob/master/testcrash_python_threadlocal_py.py The Python code also contains some code path with a workaround which I'm using currently to avoid such crashes in my application. -- components: Interpreter Core messages: 182577 nosy: Albert.Zeyer priority: normal severity: normal status: open title: crash when tp_dealloc allows other threads type: crash versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17263 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15885] @staticmethod __getattr__ doesn't work
Albert Zeyer added the comment:
I don't quite understand. Shouldn't __getattr__ also work in old-style classes?
And the error itself ('staticmethod' object is not callable), shouldn't that be
impossible?
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15885
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15885] @staticmethod __getattr__ doesn't work
New submission from Albert Zeyer:
Code:
```
class Wrapper:
@staticmethod
def __getattr__(item):
return repr(item) # dummy
a = Wrapper()
print(a.foo)
```
Expected output: 'foo'
Actual output with Python 2.7:
Traceback (most recent call last):
File test_staticmethodattr.py, line 7, in module
print(a.foo)
TypeError: 'staticmethod' object is not callable
Python 3.2 does return the expected ('foo').
PyPy returns the expected 'foo'.
--
components: Interpreter Core
messages: 170070
nosy: Albert.Zeyer
priority: normal
severity: normal
status: open
title: @staticmethod __getattr__ doesn't work
versions: Python 2.7
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15885
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14658] Overwriting dict.__getattr__ is inconsistent
New submission from Albert Zeyer alb...@googlemail.com:
```
class Foo1(dict):
def __getattr__(self, key): return self[key]
def __setattr__(self, key, value): self[key] = value
class Foo2(dict):
__getattr__ = dict.__getitem__
__setattr__ = dict.__setitem__
o1 = Foo1()
o1.x = 42
print(o1, o1.x)
o2 = Foo2()
o2.x = 42
print(o2, o2.x)
```
With CPython 2.5, 2.6 (similarly in 3.2), I get:
({'x': 42}, 42)
({}, 42)
With PyPy 1.5.0, I get the expected output::
({'x': 42}, 42)
({'x': 42}, 42)
I asked this also on SO:
http://stackoverflow.com/questions/6305267/python-inconsistence-in-the-way-you-define-the-function-setattr
From the answers, I am not exactly sure wether this is considered as a bug in
CPython or not. Anyway, I just wanted to post this here.
--
components: None
messages: 159099
nosy: Albert.Zeyer
priority: normal
severity: normal
status: open
title: Overwriting dict.__getattr__ is inconsistent
type: behavior
versions: Python 2.7
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14658
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12861] PyOS_Readline uses single lock
Albert Zeyer alb...@googlemail.com added the comment: You might have opened several via `openpty`. I am doing that here: https://github.com/albertz/PyTerminal -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12861 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12869] PyOS_StdioReadline is printing the prompt on stderr
New submission from Albert Zeyer alb...@googlemail.com: PyOS_StdioReadline from Parser/myreadline.c is printing the prompt on stderr. I think it should print it on the given parameter sys_stdout. Other readline implementations (like from the readline module) also behave this way. Even if it really is supposed to write on stderr, it should use the `sys.stderr` and not the system stderr. -- messages: 143256 nosy: Albert.Zeyer priority: normal severity: normal status: open title: PyOS_StdioReadline is printing the prompt on stderr versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12869 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12861] PyOS_Readline uses single lock
Albert Zeyer alb...@googlemail.com added the comment: Even more problematic: The readline lib itself is absolutely not designed in a way to be used from multi threads at once. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12861 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12859] readline implementation doesn't release the GIL
New submission from Albert Zeyer alb...@googlemail.com: Modules/readline.c 's `call_readline` doesn't release the GIL while reading. -- messages: 143226 nosy: Albert.Zeyer priority: normal severity: normal status: open title: readline implementation doesn't release the GIL versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12859 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12859] readline implementation doesn't release the GIL
Albert Zeyer alb...@googlemail.com added the comment: Whoops, sorry, invalid. It doesn't need to. It is handled in PyOS_Readline. -- resolution: - invalid status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12859 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12861] PyOS_Readline uses single lock
New submission from Albert Zeyer alb...@googlemail.com: In Parser/myreadline.c PyOS_Readline uses a single lock (`_PyOS_ReadlineLock`). I guess it is so that we don't have messed up stdin reads. Or are there other technical reasons? However, it should work to call this function from multiple threads with different/independent `sys_stdin` / `sys_stdout`. -- messages: 143229 nosy: Albert.Zeyer priority: normal severity: normal status: open title: PyOS_Readline uses single lock versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12861 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12861] PyOS_Readline uses single lock
Albert Zeyer alb...@googlemail.com added the comment: Ok, it seems that the Modules/readline.c implementation is also not really threadsafe... (Whereby, I think it should be.) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12861 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12854] PyOS_Readline usage in tokenizer ignores sys.stdin/sys.stdout
New submission from Albert Zeyer alb...@googlemail.com: In Parser/tokenizer.c, there is `PyOS_Readline(stdin, stdout, tok-prompt)`. This ignores any `sys.stdin` / `sys.stdout` overwrites. The usage should be like in Python/bltinmodule.c in builtin_raw_input. -- messages: 143168 nosy: Albert.Zeyer priority: normal severity: normal status: open title: PyOS_Readline usage in tokenizer ignores sys.stdin/sys.stdout versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12854 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12608] crash in PyAST_Compile when running Python code
New submission from Albert Zeyer alb...@googlemail.com:
Code:
```
import ast
globalsDict = {}
fAst = ast.FunctionDef(
name=foo,
args=ast.arguments(
args=[], vararg=None, kwarg=None, defaults=[],
kwonlyargs=[], kw_defaults=[]),
body=[], decorator_list=[])
exprAst = ast.Interactive(body=[fAst])
ast.fix_missing_locations(exprAst)
compiled = compile(exprAst, foo, single)
eval(compiled, globalsDict, globalsDict)
print(globalsDict[foo])
```
Also CPython 2.6, 2.7, 3.0 and PyPy 1.5 crashes on this.
--
components: Interpreter Core
messages: 140873
nosy: Albert.Zeyer
priority: normal
severity: normal
status: open
title: crash in PyAST_Compile when running Python code
versions: Python 3.2
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12608
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12608] crash in PyAST_Compile when running Python code
Albert Zeyer alb...@googlemail.com added the comment: PyPy bug report: https://bugs.pypy.org/issue806 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12608 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12609] SystemError: Objects/codeobject.c:64: bad argument to internal function
New submission from Albert Zeyer alb...@googlemail.com:
Code:
```
from ast import *
globalsDict = {}
exprAst = Interactive(body=[FunctionDef(name=u'Py_Main',
args=arguments(args=[Name(id=u'argc', ctx=Param()), Name(id=u'argv',
ctx=Param())], vararg=None, kwarg=None, defaults=[]),
body=[Assign(targets=[Name(id=u'argc', ctx=Store())],
value=Call(func=Attribute(value=Name(id='ctypes', ctx=Load()), attr='c_int',
ctx=Load()), args=[Attribute(value=Name(id=u'argc', ctx=Load()), attr='value',
ctx=Load())], keywords=[], starargs=None, kwargs=None)),
Assign(targets=[Name(id=u'argv', ctx=Store())],
value=Call(func=Attribute(value=Name(id='ctypes', ctx=Load()), attr='cast',
ctx=Load()), args=[Call(func=Attribute(value=Name(id='ctypes', ctx=Load()),
attr='c_void_p', ctx=Load()),
args=[Attribute(value=Call(func=Attribute(value=Name(id='ctypes', ctx=Load()),
attr='cast', ctx=Load()), args=[Name(id=u'argv', ctx=Load()),
Attribute(value=Name(id='ctypes', ctx=Load()), attr='c_void_p', ctx=Load())],
keywords=[], starargs=None, kwargs=None), attr='value', ctx=Load())],
keywords=[], starargs=No
ne, kwargs=None), Call(func=Attribute(value=Name(id='ctypes', ctx=Load()),
attr='POINTER', ctx=Load()), args=[Call(func=Attribute(value=Name(id='ctypes',
ctx=Load()), attr='POINTER', ctx=Load()),
args=[Attribute(value=Name(id='ctypes', ctx=Load()), attr='c_char',
ctx=Load())], keywords=[], starargs=None, kwargs=None)], keywords=[],
starargs=None, kwargs=None)], keywords=[], starargs=None, kwargs=None)),
Assign(targets=[Name(id=u'c', ctx=Store())],
value=Call(func=Attribute(value=Name(id='ctypes', ctx=Load()), attr='c_int',
ctx=Load()), args=[], keywords=[], starargs=None, kwargs=None)),
Assign(targets=[Name(id=u'sts', ctx=Store())],
value=Call(func=Attribute(value=Name(id='ctypes', ctx=Load()), attr='c_int',
ctx=Load()), args=[], keywords=[], starargs=None, kwargs=None)),
Assign(targets=[Name(id=u'command', ctx=Store())],
value=Call(func=Attribute(value=Name(id='ctypes', ctx=Load()), attr='cast',
ctx=Load()), args=[Call(func=Attribute(value=Name(id='ctypes', ctx=Load()), att
r='c_void_p', ctx=Load()),
args=[Attribute(value=Call(func=Attribute(value=Name(id='ctypes', ctx=Load()),
attr='c_uint', ctx=Load()), args=[Num(n=0L)], keywords=[], starargs=None,
kwargs=None), attr='value', ctx=Load())], keywords=[], starargs=None,
kwargs=None), Call(func=Attribute(value=Name(id='ctypes', ctx=Load()),
attr='POINTER', ctx=Load()), args=[Attribute(value=Name(id='ctypes',
ctx=Load()), attr='c_char', ctx=Load())], keywords=[], starargs=None,
kwargs=None)], keywords=[], starargs=None, kwargs=None)),
Assign(targets=[Name(id=u'filename', ctx=Store())],
value=Call(func=Attribute(value=Name(id='ctypes', ctx=Load()), attr='cast',
ctx=Load()), args=[Call(func=Attribute(value=Name(id='ctypes', ctx=Load()),
attr='c_void_p', ctx=Load()),
args=[Attribute(value=Call(func=Attribute(value=Name(id='ctypes', ctx=Load()),
attr='c_uint', ctx=Load()), args=[Num(n=0L)], keywords=[], starargs=None,
kwargs=None), attr='value', ctx=Load())], keywords=[], starargs=None,
kwargs=None),
Call(func=Attribute(value=Name(id='ctypes', ctx=Load()), attr='POINTER',
ctx=Load()), args=[Attribute(value=Name(id='ctypes', ctx=Load()),
attr='c_char', ctx=Load())], keywords=[], starargs=None, kwargs=None)],
keywords=[], starargs=None, kwargs=None)), Assign(targets=[Name(id=u'module',
ctx=Store())], value=Call(func=Attribute(value=Name(id='ctypes', ctx=Load()),
attr='cast', ctx=Load()), args=[Call(func=Attribute(value=Name(id='ctypes',
ctx=Load()), attr='c_void_p', ctx=Load()),
args=[Attribute(value=Call(func=Attribute(value=Name(id='ctypes', ctx=Load()),
attr='c_uint', ctx=Load()), args=[Num(n=0L)], keywords=[], starargs=None,
kwargs=None), attr='value', ctx=Load())], keywords=[], starargs=None,
kwargs=None), Call(func=Attribute(value=Name(id='ctypes', ctx=Load()),
attr='POINTER', ctx=Load()), args=[Attribute(value=Name(id='ctypes',
ctx=Load()), attr='c_char', ctx=Load())], keywords=[], starargs=None,
kwargs=None)], keywords=[], starargs=None, kwargs=None)), Assign(targe
ts=[Name(id=u'fp', ctx=Store())],
value=Call(func=Attribute(value=Name(id='ctypes', ctx=Load()), attr='cast',
ctx=Load()), args=[Call(func=Attribute(value=Name(id='ctypes', ctx=Load()),
attr='c_void_p', ctx=Load()), args=[Num(n=0)], keywords=[], starargs=None,
kwargs=None), Call(func=Attribute(value=Name(id='ctypes', ctx=Load()),
attr='POINTER', ctx=Load()), args=[Attribute(value=Name(id='ctypes',
ctx=Load()), attr='c_int', ctx=Load())], keywords=[], starargs=None,
kwargs=None)], keywords=[], starargs=None, kwargs=None)),
Assign(targets=[Name(id=u'p', ctx=Store())],
value=Call(func=Call(func=Attribute(value=Name(id='ctypes', ctx=Load()),
attr='POINTER', ctx=Load()), args=[Attribute(value=Name(id='ctypes',
ctx=Load()), attr='c_char', ctx=Load())], keywords=[], starargs=None,
kwargs=None), args=[], keywords=[], starargs=None, kwargs=None)),
Assign
[issue12610] Fatal Python error: non-string found in code slot
New submission from Albert Zeyer alb...@googlemail.com:
Code:
```
from ast import *
globalsDict = {}
body = [
Assign(targets=[Name(id=u'argc', ctx=Store())],
value=Name(id=u'None', ctx=Load())),
]
exprAst = Interactive(body=[
FunctionDef(
name='foo',
args=arguments(args=[Name(id=u'argc', ctx=Param()),
Name(id=u'argv', ctx=Param())],
vararg=None, kwarg=None,
defaults=[]),
body=body,
decorator_list=[])])
fix_missing_locations(exprAst)
compiled = compile(exprAst, foo, single)
eval(compiled, {}, globalsDict)
f = globalsDict[foo]
print(f)
```
CPython 2.7.1: Fatal Python error: non-string found in code slot
PyPy 1.5: function foo at 0x000103114430
--
messages: 140877
nosy: Albert.Zeyer
priority: normal
severity: normal
status: open
title: Fatal Python error: non-string found in code slot
versions: Python 2.7
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12610
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12609] SystemError: Objects/codeobject.c:64: bad argument to internal function
Albert Zeyer alb...@googlemail.com added the comment:
Simplified code:
```
from ast import *
globalsDict = {}
exprAst = Interactive(body=[
FunctionDef(
name=u'foo',
args=arguments(args=[], vararg=None, kwarg=None, defaults=[]),
body=[Pass()],
decorator_list=[])])
fix_missing_locations(exprAst)
compiled = compile(exprAst, foo, single)
eval(compiled, {}, globalsDict)
f = globalsDict[foo]
print(f)
```
If I change `name=u'foo'` to `name='foo'`, it works.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12609
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12311] memory leak with self-referencing dict
New submission from Albert Zeyer alb...@googlemail.com: The attached Python script leaks memory. It is clear that there is a reference circle (`__dict__` references `self`) but `gc.collect()` should find this. -- components: Interpreter Core files: py_dict_refcount_test.py messages: 138062 nosy: Albert.Zeyer priority: normal severity: normal status: open title: memory leak with self-referencing dict type: resource usage versions: Python 3.2 Added file: http://bugs.python.org/file22311/py_dict_refcount_test.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12311 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12311] memory leak with self-referencing dict
Albert Zeyer alb...@googlemail.com added the comment: Whoops, looks like a duplicate of #1469629. -- resolution: - duplicate status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12311 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
