[Python-Dev] Add function to signal module for getting main thread id
Main thread is slightly different from others. Signals can be subscribed from main thread only. Tulip has special logic for main thread. In application code we can explicitly know which thread is executed, main or not. But from library it's not easy. Tulip uses check like threading.current_thread().name == 'MainThread' This approach has a problem: thread name is writable attribute and can be changed by user code. My proposition is to add function like get_mainthread_id() - int which return ident for main thread (I know function name is not perfect, please guess better one). Signal module already has required data as internal variable static long main_thread; I just guess to expose this value to python. Thoughts? -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add function to signal module for getting main thread id
Le Fri, 30 Aug 2013 12:24:07 +0300, Andrew Svetlov andrew.svet...@gmail.com a écrit : Main thread is slightly different from others. Signals can be subscribed from main thread only. Tulip has special logic for main thread. In application code we can explicitly know which thread is executed, main or not. But from library it's not easy. Tulip uses check like threading.current_thread().name == 'MainThread' This approach has a problem: thread name is writable attribute and can be changed by user code. Really? Please at least use: My proposition is to add function like get_mainthread_id() - int which return ident for main thread (I know function name is not perfect, please guess better one). Signal module already has required data as internal variable static long main_thread; I just guess to expose this value to python. Thoughts? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add function to signal module for getting main thread id
2013/8/30 Andrew Svetlov andrew.svet...@gmail.com: Tulip uses check like threading.current_thread().name == 'MainThread' You should use the identifier, not the name: threading.current_thread().ident. This approach has a problem: thread name is writable attribute and can be changed by user code. The ident attribute cannot be modified. My proposition is to add function like get_mainthread_id() - int which return ident for main thread (I know function name is not perfect, please guess better one). Just call threading.get_ident() at startup, when you have only one thread (the main thread). There is an ugly hack to get the identifier of the main thread: threading._shutdown.__self__.ident. Signal module already has required data as internal variable static long main_thread; I just guess to expose this value to python. Thoughts? If we expose the identifier of the main thread, something should be added to the threading module, not the signal module. Is it possible that the main thread exit while there are still other live threads? Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add function to signal module for getting main thread id
Le Fri, 30 Aug 2013 12:24:07 +0300, Andrew Svetlov andrew.svet...@gmail.com a écrit : Main thread is slightly different from others. Signals can be subscribed from main thread only. Tulip has special logic for main thread. In application code we can explicitly know which thread is executed, main or not. But from library it's not easy. Tulip uses check like threading.current_thread().name == 'MainThread' This approach has a problem: thread name is writable attribute and can be changed by user code. Please at least use: isinstance(threading.current_thread(), threading._MainThread) True But really, what we need is a threading.main_thread() function. (Apologies for the previous incomplete reply (keyboard mishap)) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add function to signal module for getting main thread id
Le Fri, 30 Aug 2013 11:36:57 +0200, Victor Stinner victor.stin...@gmail.com a écrit : If we expose the identifier of the main thread, something should be added to the threading module, not the signal module. Agreed. Is it possible that the main thread exit while there are still other live threads? exit in what sense? In the C sense, no: when the main C thread exits, the whole process is terminated (this is how our daemon threads work). In the Python sense, yes: we have a test for it: http://hg.python.org/cpython/file/c347b9063a9e/Lib/test/test_threading.py#l325 Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add function to signal module for getting main thread id
On Fri, Aug 30, 2013 at 12:24:07PM +0300, Andrew Svetlov andrew.svet...@gmail.com wrote: Main thread is slightly different from others. Signals can be subscribed from main thread only. Tulip has special logic for main thread. In application code we can explicitly know which thread is executed, main or not. But from library it's not easy. Tulip uses check like threading.current_thread().name == 'MainThread' This approach has a problem: thread name is writable attribute and can be changed by user code. You can test threading.current_thread().__class__ is threading._MainThread or threading.current_thread().ident == threading._MainThread.ident My proposition is to add function like get_mainthread_id() - int which return ident for main thread threading._MainThread.ident ? Oleg. -- Oleg Broytmanhttp://phdru.name/p...@phdru.name Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add function to signal module for getting main thread id
I missed _MainThread in threading, that's why I've guessed to add function to signal module. threading.main_thread() is much better sure. On Fri, Aug 30, 2013 at 12:39 PM, Antoine Pitrou solip...@pitrou.net wrote: Le Fri, 30 Aug 2013 12:24:07 +0300, Andrew Svetlov andrew.svet...@gmail.com a écrit : Main thread is slightly different from others. Signals can be subscribed from main thread only. Tulip has special logic for main thread. In application code we can explicitly know which thread is executed, main or not. But from library it's not easy. Tulip uses check like threading.current_thread().name == 'MainThread' This approach has a problem: thread name is writable attribute and can be changed by user code. Please at least use: isinstance(threading.current_thread(), threading._MainThread) True But really, what we need is a threading.main_thread() function. (Apologies for the previous incomplete reply (keyboard mishap)) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add function to signal module for getting main thread id
_MainThread can be used as workaround, but adding public function makes value. Oleg, as I understand _MainThread is a class, not class instance, test for threading._MainThread.ident doesn't make sense. On Fri, Aug 30, 2013 at 12:44 PM, Oleg Broytman p...@phdru.name wrote: On Fri, Aug 30, 2013 at 12:24:07PM +0300, Andrew Svetlov andrew.svet...@gmail.com wrote: Main thread is slightly different from others. Signals can be subscribed from main thread only. Tulip has special logic for main thread. In application code we can explicitly know which thread is executed, main or not. But from library it's not easy. Tulip uses check like threading.current_thread().name == 'MainThread' This approach has a problem: thread name is writable attribute and can be changed by user code. You can test threading.current_thread().__class__ is threading._MainThread or threading.current_thread().ident == threading._MainThread.ident My proposition is to add function like get_mainthread_id() - int which return ident for main thread threading._MainThread.ident ? Oleg. -- Oleg Broytmanhttp://phdru.name/p...@phdru.name Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add function to signal module for getting main thread id
I've filed http://bugs.python.org/issue18882 for this. On Fri, Aug 30, 2013 at 12:52 PM, Andrew Svetlov andrew.svet...@gmail.com wrote: _MainThread can be used as workaround, but adding public function makes value. Oleg, as I understand _MainThread is a class, not class instance, test for threading._MainThread.ident doesn't make sense. On Fri, Aug 30, 2013 at 12:44 PM, Oleg Broytman p...@phdru.name wrote: On Fri, Aug 30, 2013 at 12:24:07PM +0300, Andrew Svetlov andrew.svet...@gmail.com wrote: Main thread is slightly different from others. Signals can be subscribed from main thread only. Tulip has special logic for main thread. In application code we can explicitly know which thread is executed, main or not. But from library it's not easy. Tulip uses check like threading.current_thread().name == 'MainThread' This approach has a problem: thread name is writable attribute and can be changed by user code. You can test threading.current_thread().__class__ is threading._MainThread or threading.current_thread().ident == threading._MainThread.ident My proposition is to add function like get_mainthread_id() - int which return ident for main thread threading._MainThread.ident ? Oleg. -- Oleg Broytmanhttp://phdru.name/p...@phdru.name Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] EINTR handling...
Hello, This has been bothering me for years: why don't we properly handle EINTR, by running registered signal handlers and restarting the interrupted syscall (or eventually returning early e.g. for sleep)? EINTR is really a nuisance, and exposing it to Python code is just pointless. Now some people might argue that some code relies on EINTR to interrupt a syscall on purpose, but I don't really buy it: it's highly non-portable (depends on the syscall, SA_RESTART flag...) and subject to race conditions (it it comes before the syscall or if you get a partial read/write you'll deadlock). Furthermore, the stdlib code base is not consistent: some code paths handle EINTR, e.g. subprocess, multiprocessing, sock_sendall() does but not sock_send()... Just grep for EINTR and InterruptedError and you'll be amazed. GHC, the JVM and probably other platforms handle EINTR, maybe it's time for us too? Just for reference, here are some issues due to EINTR popping up: http://bugs.python.org/issue17097 http://bugs.python.org/issue12268 http://bugs.python.org/issue9867 http://bugs.python.org/issue7978 http://bugs.python.org/issue12493 http://bugs.python.org/issue3771 cf ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] EINTR handling...
2013/8/30 Charles-François Natali cf.nat...@gmail.com Hello, This has been bothering me for years: why don't we properly handle EINTR, by running registered signal handlers and restarting the interrupted syscall (or eventually returning early e.g. for sleep)? EINTR is really a nuisance, and exposing it to Python code is just pointless. I agree. Is there a way to see in C code where EINTR is not handled? Or a method to handle this systematically? Now some people might argue that some code relies on EINTR to interrupt a syscall on purpose, but I don't really buy it: it's highly non-portable (depends on the syscall, SA_RESTART flag...) and subject to race conditions (it it comes before the syscall or if you get a partial read/write you'll deadlock). Furthermore, the stdlib code base is not consistent: some code paths handle EINTR, e.g. subprocess, multiprocessing, sock_sendall() does but not sock_send()... Just grep for EINTR and InterruptedError and you'll be amazed. GHC, the JVM and probably other platforms handle EINTR, maybe it's time for us too? Just for reference, here are some issues due to EINTR popping up: http://bugs.python.org/issue17097 http://bugs.python.org/issue12268 http://bugs.python.org/issue9867 http://bugs.python.org/issue7978 http://bugs.python.org/issue12493 http://bugs.pythoto see n.org/issue3771 http://bugs.python.org/issue3771 cf ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/amauryfa%40gmail.com -- Amaury Forgeot d'Arc ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add function to signal module for getting main thread id
Am 30.08.2013 11:39, schrieb Antoine Pitrou: Le Fri, 30 Aug 2013 12:24:07 +0300, Andrew Svetlov andrew.svet...@gmail.com a écrit : Main thread is slightly different from others. Signals can be subscribed from main thread only. Tulip has special logic for main thread. In application code we can explicitly know which thread is executed, main or not. But from library it's not easy. Tulip uses check like threading.current_thread().name == 'MainThread' This approach has a problem: thread name is writable attribute and can be changed by user code. Please at least use: isinstance(threading.current_thread(), threading._MainThread) True But really, what we need is a threading.main_thread() function. What happens, when a program fork()s from another thread than the main thread? AFAIR the other threads are suspended and the forking thread is the new main thread. Or something similar... (Yes, I'm aware that threading + fork is an abomination.) Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add function to signal module for getting main thread id
On 30 August 2013 20:27, Andrew Svetlov andrew.svet...@gmail.com wrote: I've filed http://bugs.python.org/issue18882 for this. I don't actually object to the addition, but is there any way that threading.enumerate()[0] *won't* be the main thread? (subinterpreters, perhaps, but they're going to have trouble anyway, since they won't have access to the real main thread) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] EINTR handling...
On 30 August 2013 20:29, Charles-François Natali cf.nat...@gmail.com wrote: Hello, This has been bothering me for years: why don't we properly handle EINTR, by running registered signal handlers and restarting the interrupted syscall (or eventually returning early e.g. for sleep)? EINTR is really a nuisance, and exposing it to Python code is just pointless. Now some people might argue that some code relies on EINTR to interrupt a syscall on purpose, but I don't really buy it: it's highly non-portable (depends on the syscall, SA_RESTART flag...) and subject to race conditions (it it comes before the syscall or if you get a partial read/write you'll deadlock). Furthermore, the stdlib code base is not consistent: some code paths handle EINTR, e.g. subprocess, multiprocessing, sock_sendall() does but not sock_send()... Just grep for EINTR and InterruptedError and you'll be amazed. GHC, the JVM and probably other platforms handle EINTR, maybe it's time for us too? Sounds good to me. I don't believe there's been a conscious decision that we *shouldn't* handle it, it just hasn't annoyed anyone enough for them to propose a systematic fix in CPython. If that latter part is no longer true, great ;) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Coverity Scan Spotlight Python
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Am 30.08.2013 01:24, schrieb Sturla Molden: Do the numbers add up? .005 defects in 1,000 lines of code is one defect in every 200,000 lines of code. However they also claim that to date, the Coverity Scan service has analyzed nearly 400,000 lines of Python code and identified 996 new defects – 860 of which have been fixed by the Python community. Yes, the numbers add up. The difference between 860 and 996 are false positive defects and code that is intentionally written in a way, which looks suspicious to Coverity Scan. I have documented the most common limitations in the devguide [1]. By the way Coverity Scan doesn't understand Python code. It can only analyzes C, C++ and Java code. [1] Christian -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iQIcBAEBCgAGBQJSII1+AAoJEMeIxMHUVQ1FNsIP/jmiMD8p39zHj8Ggb6NM1q9W WotnQzM2vLE90s9VfewQB914u4rFjEtYVWD6P88QQEwdcrBfnMs/xvFBcyW/yuVd EB57hTjqWKSgGdcFsKoAmlFtSzFTUtM3Yc4aiyYHwsn7vJPTbxAO/6GAToGhHeP6 96f0oXz4uqeM4RJNCbHPt57kHT9OUhsITiZ11rtlsYziGwpRKL5K7bd+bbh/HlPy BDRVfU112vDjOiCRFGPlmMy2ShJabZwT5uZ4+0VGgGo5/Af3H3UU7pYw1cuwnjgh CIv/jYFH8OgNvC+hwvai2OxQfH7aXtUhcSPUSOOmPUQ/pbkTMY65Ya2iIRtEoIrY 8FwayYTMzGkCkEZoS4HXO1wGNCcj3tM8ivGP89aJDpySYLmuJoLa5x/aNKKxyo+X n9HT4BAkuYuFi1qQsPh9kW+FR4VCWTob7BSjOXrY7T8X6plon+fwFseQMkE8JUqI ckwTJCHDIc23d/HiTNhI8Ank3v28JQLdVTIPYnSKU6YpxjDAO0J+BgExAHpAyVwZ snEz9zVj/x4YRkUgxWwTMj/ctKDEpX9mehg5rytlWIaKUtPbTmR+aWxG06+TCd1c dg0cEYso+tvVUAYfZX24dn/7NPrmkBHjGM0ph2PH0S+GcpHF861GvflaSwzQ/ceD kYF3msFihRocFXfy8iNj =Usp8 -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add function to signal module for getting main thread id
Le Fri, 30 Aug 2013 22:09:37 +1000, Nick Coghlan ncogh...@gmail.com a écrit : On 30 August 2013 20:27, Andrew Svetlov andrew.svet...@gmail.com wrote: I've filed http://bugs.python.org/issue18882 for this. I don't actually object to the addition, but is there any way that threading.enumerate()[0] *won't* be the main thread? enumerate() doesn't guarantee any ordering, and the underlying container is a dict (actually, there are two of them). (subinterpreters, perhaps, but they're going to have trouble anyway, since they won't have access to the real main thread) Ah, subinterpreters :-) cheers Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add function to signal module for getting main thread id
Le Fri, 30 Aug 2013 14:06:11 +0200, Christian Heimes christ...@python.org a écrit : Am 30.08.2013 11:39, schrieb Antoine Pitrou: Le Fri, 30 Aug 2013 12:24:07 +0300, Andrew Svetlov andrew.svet...@gmail.com a écrit : Main thread is slightly different from others. Signals can be subscribed from main thread only. Tulip has special logic for main thread. In application code we can explicitly know which thread is executed, main or not. But from library it's not easy. Tulip uses check like threading.current_thread().name == 'MainThread' This approach has a problem: thread name is writable attribute and can be changed by user code. Please at least use: isinstance(threading.current_thread(), threading._MainThread) True But really, what we need is a threading.main_thread() function. What happens, when a program fork()s from another thread than the main thread? AFAIR the other threads are suspended and the forking thread is the new main thread. Or something similar... Yes. We even support it :-) http://hg.python.org/cpython/file/c347b9063a9e/Lib/test/test_threading.py#l503 (well, whoever wrote that test wanted to support it. I don't think that's me) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add function to signal module for getting main thread id
I've made a patch. It works except scenario described by Christian Heimes. See details in http://bugs.python.org/issue18882 On Fri, Aug 30, 2013 at 3:21 PM, Antoine Pitrou solip...@pitrou.net wrote: Le Fri, 30 Aug 2013 14:06:11 +0200, Christian Heimes christ...@python.org a écrit : Am 30.08.2013 11:39, schrieb Antoine Pitrou: Le Fri, 30 Aug 2013 12:24:07 +0300, Andrew Svetlov andrew.svet...@gmail.com a écrit : Main thread is slightly different from others. Signals can be subscribed from main thread only. Tulip has special logic for main thread. In application code we can explicitly know which thread is executed, main or not. But from library it's not easy. Tulip uses check like threading.current_thread().name == 'MainThread' This approach has a problem: thread name is writable attribute and can be changed by user code. Please at least use: isinstance(threading.current_thread(), threading._MainThread) True But really, what we need is a threading.main_thread() function. What happens, when a program fork()s from another thread than the main thread? AFAIR the other threads are suspended and the forking thread is the new main thread. Or something similar... Yes. We even support it :-) http://hg.python.org/cpython/file/c347b9063a9e/Lib/test/test_threading.py#l503 (well, whoever wrote that test wanted to support it. I don't think that's me) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] EINTR handling...
2013/8/30 Amaury Forgeot d'Arc amaur...@gmail.com:
I agree.
Is there a way to see in C code where EINTR is not handled?
EINTR can be returned on slow syscalls, so a good heuristic would be
to start with code that releases the GIL.
But I don't see a generic way apart from grepping for syscalls that
are documented to return EINTR.
Or a method to handle this systematically?
The glibc defines this macro:
# define TEMP_FAILURE_RETRY(expression) \
(__extension__ \
({ long int __result; \
do __result = (long int) (expression); \
while (__result == -1L errno == EINTR); \
__result; }))
#endif
which you can then use as:
pid = TEMP_FAILURE_RETRY(waitpid(pid, status, options));
Unfortunately, it's not as easy for us, since we must release the GIL
around the syscall, try again if it failed with EINTR, only after
having called PyErr_CheckSignals() to run signal handlers.
e.g. waitpid():
Py_BEGIN_ALLOW_THREADS
pid = waitpid(pid, status, options);
Py_END_ALLOW_THREADS
should become (conceptually):
begin_handle_eintr:
Py_BEGIN_ALLOW_THREADS
pid = waitpid(pid, status, options);
Py_END_ALLOW_THREADS
if (pid 0 errno == EINTR) {
if (PyErr_CheckSignals())
return NULL;
goto begin_handle_eintr;
}
We might want to go for a clever macro (like BEGIN_SELECT_LOOP in
socketmodule.c).
2013/8/30 Nick Coghlan ncogh...@gmail.com:
Sounds good to me. I don't believe there's been a conscious decision
that we *shouldn't* handle it, it just hasn't annoyed anyone enough
for them to propose a systematic fix in CPython. If that latter part
is no longer true, great ;)
Great, I'll open a bug report then :)
cf
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] EINTR handling...
On Fri, 30 Aug 2013 12:29:12 +0200 Charles-François Natali cf.nat...@gmail.com wrote: Furthermore, the stdlib code base is not consistent: some code paths handle EINTR, e.g. subprocess, multiprocessing, sock_sendall() does but not sock_send()... Just grep for EINTR and InterruptedError and you'll be amazed. GHC, the JVM and probably other platforms handle EINTR, maybe it's time for us too? I don't have any precise opinion on this. It's true that we should have a systematic approach, I just don't know if all interfaces should handler EINTR automatically, or only the high-level ones. (for the sake of clarity, I'm fine with either :-)) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Issue #17741: Rename IncrementalParser and its methods.
Hello, On Fri, 30 Aug 2013 14:51:42 +0200 (CEST) eli.bendersky python-check...@python.org wrote: diff --git a/Doc/library/xml.etree.elementtree.rst b/Doc/library/xml.etree.elementtree.rst --- a/Doc/library/xml.etree.elementtree.rst +++ b/Doc/library/xml.etree.elementtree.rst @@ -105,37 +105,42 @@ root[0][1].text '2008' -Incremental parsing -^^^ +Pull API for asynchronous parsing +^ I think the documentation should use another term than asynchronous: non-blocking or event-driven would be fine. Asynchronous is loaded with various meanings and connotations which IMO make it the wrong term here. For example, in many contexts asynchronous means does the job behind your back, e.g. in a worker thread. POSIX defines some asynchronous I/O APIs which are ostensibly not the same as non-blocking I/O: http://pubs.opengroup.org/onlinepubs/9699919799/functions/aio_read.html Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Issue #17741: Rename IncrementalParser and its methods.
On Fri, Aug 30, 2013 at 10:07 AM, Antoine Pitrou solip...@pitrou.netwrote: Hello, On Fri, 30 Aug 2013 14:51:42 +0200 (CEST) eli.bendersky python-check...@python.org wrote: diff --git a/Doc/library/xml.etree.elementtree.rst b/Doc/library/xml.etree.elementtree.rst --- a/Doc/library/xml.etree.elementtree.rst +++ b/Doc/library/xml.etree.elementtree.rst @@ -105,37 +105,42 @@ root[0][1].text '2008' -Incremental parsing -^^^ +Pull API for asynchronous parsing +^ I think the documentation should use another term than asynchronous: non-blocking or event-driven would be fine. Asynchronous is loaded with various meanings and connotations which IMO make it the wrong term here. For example, in many contexts asynchronous means does the job behind your back, e.g. in a worker thread. POSIX defines some asynchronous I/O APIs which are ostensibly not the same as non-blocking I/O: http://pubs.opengroup.org/onlinepubs/9699919799/functions/aio_read.html Makes sense. I'll change it to non-blocking, since this doc already uses blocking here and there to refer to the opposite effect. Eli ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] EINTR handling...
I don't have a strong opinion on this either. The distinction between send() and send_all() makes sense to me though (send_all() works hard to get all your data out, send() only does what it can quickly). Personally for calls like select() I think returning early on EINTR makes sense, it's usually part of a select loop anyway. The only thing I care deeply about is that you shouldn't restart anything before letting a Python-level signal handler run. A program might well have a Python signal handler that must run before the I/O is restarted, and the signal handler might raise an exception (like the default SIGINT handler, which raises KeyboardInterrupt). On Fri, Aug 30, 2013 at 10:02 AM, Antoine Pitrou solip...@pitrou.netwrote: On Fri, 30 Aug 2013 12:29:12 +0200 Charles-François Natali cf.nat...@gmail.com wrote: Furthermore, the stdlib code base is not consistent: some code paths handle EINTR, e.g. subprocess, multiprocessing, sock_sendall() does but not sock_send()... Just grep for EINTR and InterruptedError and you'll be amazed. GHC, the JVM and probably other platforms handle EINTR, maybe it's time for us too? I don't have any precise opinion on this. It's true that we should have a systematic approach, I just don't know if all interfaces should handler EINTR automatically, or only the high-level ones. (for the sake of clarity, I'm fine with either :-)) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Coverity Scan Spotlight Python
On 8/30/2013 8:18 AM, Christian Heimes wrote: By the way Coverity Scan doesn't understand Python code. It can only analyzes C, C++ and Java code. Have you (or Coverity) thought about which, if any, of the C defect categories apply to Python? (Assuming no use of ctypes ;-). Would it make any sense to apply their technology to Python code scanning? -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2013-08-23 - 2013-08-30) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open4182 (+14) closed 26476 (+50) total 30658 (+64) Open issues with patches: 1910 Issues opened (48) == #17400: ipaddress should make it easy to identify rfc6598 addresses http://bugs.python.org/issue17400 reopened by ncoghlan #18822: poor proxyval() coverage in test_gdb http://bugs.python.org/issue18822 opened by pitrou #18823: Idle: use pipes instead of sockets to talk with user subproces http://bugs.python.org/issue18823 opened by terry.reedy #18824: Adding LogRecord attribute traceback http://bugs.python.org/issue18824 opened by Sworddragon #18825: Making msg optional on logging.exception() and similar variant http://bugs.python.org/issue18825 opened by Sworddragon #18826: reversed() requires a sequence - Could work on any iterator? http://bugs.python.org/issue18826 opened by dstufft #18828: urljoin behaves differently with custom and standard schemas http://bugs.python.org/issue18828 opened by mher.movsisyan #18829: csv produces confusing error message when passed a non-string http://bugs.python.org/issue18829 opened by Thibault.Kruse #18830: Remove duplicates from a result of getclasstree() http://bugs.python.org/issue18830 opened by serhiy.storchaka #18831: importlib.import_module() bypasses builtins.__import__ http://bugs.python.org/issue18831 opened by brett.cannon #18834: Add Clang to distutils to build C/C++ extensions http://bugs.python.org/issue18834 opened by Ryan.Gonzalez #18835: Add aligned memroy variants to the suite of PyMem functions/ma http://bugs.python.org/issue18835 opened by rhettinger #18837: multiprocessing.reduction is undocumented http://bugs.python.org/issue18837 opened by tpievila #18838: The order of interactive prompt and traceback on Windows http://bugs.python.org/issue18838 opened by Drekin #18840: Tutorial recommends pickle module without any warning of insec http://bugs.python.org/issue18840 opened by dstufft #18841: math.isfinite fails with Decimal sNAN http://bugs.python.org/issue18841 opened by stevenjd #18842: Add float.is_finite is_nan is_infinite to match Decimal method http://bugs.python.org/issue18842 opened by stevenjd #18843: Py_FatalError (msg=0x7f0e3b373232 bad leading pad byte) at P http://bugs.python.org/issue18843 opened by mmokrejs #18844: allow weights in random.choice http://bugs.python.org/issue18844 opened by aisaac #18845: 2.7.5-r2: Fatal Python error: Segmentation fault http://bugs.python.org/issue18845 opened by mmokrejs #18848: In unittest.TestResult .startTestRun() and .stopTestRun() meth http://bugs.python.org/issue18848 opened by py.user #18849: Failure to try another name for tempfile when directory with c http://bugs.python.org/issue18849 opened by vlad #18850: xml.etree.ElementTree accepts control chars. http://bugs.python.org/issue18850 opened by maker #18851: subprocess's Popen closes stdout/stderr filedescriptors used i http://bugs.python.org/issue18851 opened by janwijbrand #18852: site.py does not handle readline.__doc__ being None http://bugs.python.org/issue18852 opened by theller #18853: Got ResourceWarning unclosed file when running Lib/shlex.py de http://bugs.python.org/issue18853 opened by vajrasky #18854: is_multipart and walk should document their treatment of 'mess http://bugs.python.org/issue18854 opened by r.david.murray #18855: Inconsistent README filenames http://bugs.python.org/issue18855 opened by madison.may #18856: Added test coverage for calendar print functions http://bugs.python.org/issue18856 opened by madison.may #18857: urlencode of a None value uses the string 'None' http://bugs.python.org/issue18857 opened by Joshua.Johnston #18858: dummy_threading lacks threading.get_ident() equivalent http://bugs.python.org/issue18858 opened by zuo #18859: README.valgrind should mention --with-valgrind http://bugs.python.org/issue18859 opened by tim.peters #18860: Add content manager API to email package http://bugs.python.org/issue18860 opened by r.david.murray #18861: Problems with recursive automatic exception chaining http://bugs.python.org/issue18861 opened by Nikratio #18862: Implement __subclasshook__() for Finders and Loaders in import http://bugs.python.org/issue18862 opened by eric.snow #18864: Implementation for PEP 451 (importlib.machinery.ModuleSpec) http://bugs.python.org/issue18864 opened by eric.snow #18870: eval() uses latin-1 to decode str http://bugs.python.org/issue18870 opened by valhallasw #18872: platform.linux_distribution() doesn't recognize Amazon Linux http://bugs.python.org/issue18872 opened by Lorin.Hochstein #18873: Encoding detected in non-comment lines http://bugs.python.org/issue18873 opened by Paul.Bonser #18874: Add a new tracemalloc module to trace memory
Re: [Python-Dev] cpython: Issue #17741: Rename IncrementalParser and its methods.
I still think non-blocking sounds network-related... On Fri, Aug 30, 2013 at 12:23 PM, Eli Bendersky eli...@gmail.com wrote: On Fri, Aug 30, 2013 at 10:07 AM, Antoine Pitrou solip...@pitrou.netwrote: Hello, On Fri, 30 Aug 2013 14:51:42 +0200 (CEST) eli.bendersky python-check...@python.org wrote: diff --git a/Doc/library/xml.etree.elementtree.rst b/Doc/library/xml.etree.elementtree.rst --- a/Doc/library/xml.etree.elementtree.rst +++ b/Doc/library/xml.etree.elementtree.rst @@ -105,37 +105,42 @@ root[0][1].text '2008' -Incremental parsing -^^^ +Pull API for asynchronous parsing +^ I think the documentation should use another term than asynchronous: non-blocking or event-driven would be fine. Asynchronous is loaded with various meanings and connotations which IMO make it the wrong term here. For example, in many contexts asynchronous means does the job behind your back, e.g. in a worker thread. POSIX defines some asynchronous I/O APIs which are ostensibly not the same as non-blocking I/O: http://pubs.opengroup.org/onlinepubs/9699919799/functions/aio_read.html Makes sense. I'll change it to non-blocking, since this doc already uses blocking here and there to refer to the opposite effect. Eli ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com -- Ryan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Issue #17741: Rename IncrementalParser and its methods.
On 08/30/2013 06:37 PM, Ryan Gonzalez wrote: I still think non-blocking sounds network-related... Sometimes it is. And sometimes it's user-input related, or waiting on a local-pipeline related. But in all cases it means, return whatever is ready, don't block if nothing is ready. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 450 adding statistics module
Hi all, I think that PEP 450 is now ready for a PEP dictator. There have been a number of code reviews, and feedback has been taken into account. The test suite passes. I'm not aware of any unanswered issues with the code. At least two people other than myself think that the implementation is ready for a dictator, and nobody has objected. There is still on-going work on speeding up the implementation for the statistics.sum function, but that will not effect the interface or the substantially change the test suite. http://bugs.python.org/issue18606 http://www.python.org/dev/peps/pep-0450/ -- Steven ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Issue #17741: Rename IncrementalParser and its methods.
On 8/30/2013 9:37 PM, Ryan Gonzalez wrote: I still think non-blocking sounds network-related... But it isn't ;-). Gui apps routinely use event loops and/or threads or subprocesses to avoid blocking on either user input (which can come from keyboard or mouse) and maybe disk operations and calculations. For instance, if it became possible to run the test suite from Idle, it would need to be non-blocking so one could continue to edit while the tests run (15-20 min on my machine, divided by number of cores). -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Completing the email6 API changes.
If you've read my blog (eg: on planet python), you will be aware that
I dedicated August to full time email package development. At the
beginning of the month I worked out a design proposal for the remaining
API additions to the email package, dealing with handling message bodies
in a more natural way. I posted this to the email-sig, and got...well,
no objections. Barry Warsaw did review it, and told me he had no issues
with the overall design, but also had no time for a detailed review.
Since one way to see if a design holds together is to document and code
it, I decided to go ahead and do so. This resulted in a number of small
tweaks, but no major changes.
I have at this point completed the coding. You can view the whole
patch at:
http://bugs.python.org/issue18891
which also links to three layered patches that I posted as I went along,
if you prefer somewhat smaller patches.
I think it would be great if I could check this in for alpha2. Since it
is going in as an addition to the existing provisional code, the level
of review required is not as high as for non-provisional code, I think.
But I would certainly appreciate review from anyone so moved, since I
haven't gotten any yet.
Of course, if there is serious bikeshedding about the API, I won't
make alpha2, but that's fine.
The longer term goal, by the way, is to move all of this out of
provisional status for 3.5.
This code finishes the planned API additions for the email package
to bring it fully into the world of Python3 and unicode. It does not
fix the deep internals, which could be a future development direction
(but probably only after the old API has been retired, which will take
a while). But it does make it so that you can use the email package
without having to be a MIME expert. (You can't get away with *no* MIME
knowledge, but you no longer have to fuss with the details of the syntax.)
To give you the flavor of how the entire new provisional API plays
together, here's how you can build a complete message in your application:
from email.message import MIMEMessage
from email.headerregistry import Address
fullmsg = MIMEMessage()
fullmsg['To'] = Address('Foö Bar', 'f...@example.com')
fullmsg['From'] = mè m...@example.com
fullmsg['Subject'] = j'ai un problème de python.
fullmsg.set_content(et la il est monté sur moi et il commence
a m'étouffer.)
htmlmsg = MIMEMessage()
htmlmsg.set_content(pet la il est monté sur moi et il commence
a m'étouffer./pimg src='image1' /,
subtype='html')
with open('python.jpg', 'rb') as python:
htmlmsg.add_related(python.read(), 'image', 'jpg', cid='image1'
disposition='inline')
fullmsg.make_alternative()
fullmsg.attach(htmlmsg)
with open('police-report.txt') as report:
fullmsg.add_attachment(report.read(), filename='pölice-report.txt',
params=dict(wrap='flow'), headers=(
'X-Secret-Level: top',
'X-Authorization: Monty'))
Which results in:
for line in bytes(fullmsg).splitlines():
print(line)
b'To: =?utf-8?q?Fo=C3=B6?= Bar f...@example.com'
b'From: =?utf-8?q?m=C3=A8?= m...@example.com'
bSubject: j'ai un =?utf-8?q?probl=C3=A8me?= de python.
b'MIME-Version: 1.0'
b'Content-Type: multipart/mixed; boundary1710006838=='
b''
b'--===1710006838=='
b'Content-Type: multipart/alternative;
boundary1811969196=='
b''
b'--===1811969196=='
b'Content-Type: text/plain; charset=utf-8'
b'Content-Transfer-Encoding: 8bit'
b''
bet la il est mont\xc3\xa9 sur moi et il commence a m'\xc3\xa9touffer.
b''
b'--===1811969196=='
b'MIME-Version: 1.0'
b'Content-Type: multipart/related; boundary1469657937=='
b''
b'--===1469657937=='
b'Content-Type: text/html; charset=utf-8'
b'Content-Transfer-Encoding: quoted-printable'
b''
bpet la il est mont=C3=A9 sur moi et il commence a
m'=C3=A9touffer./pimg =
bsrc=3D'image1' /
b''
b'--===1469657937=='
b'MIME-Version: 1.0'
b'Content-Type: image/jpg'
b'Content-Transfer-Encoding: base64'
b'Content-Disposition: inline'
b'Content-ID: image1'
b''
b'ZmFrZSBpbWFnZSBkYXRhCg=='
b''
b'--===1469657937==--'
b'--===1811969196==--'
b'--===1710006838=='
b'MIME-Version: 1.0'
b'X-Secret-Level: top'
b'X-Authorization: Monty'
b'Content-Transfer-Encoding: 7bit'
b'Content-Disposition: attachment; filename*=utf-8''p%C3%B6lice-report.txt
b'Content-Type: text/plain; charset=utf-8; wrap=flow'
b''
b'il est sorti de son vivarium.'
b''
b'--===1710006838==--'
If you've used the email package
