[issue29859] Return code of pthread_* in thread_pthread.h is not used for perror

[INADA Naoki]

INADA Naoki added the comment:


New changeset d7fa6b259e00fca04dbf816bfcf4115fdda14bb7 by INADA Naoki (Daniel 
Birnstiel) in branch 'master':
bpo-29859: Fix error messages from return codes for pthread_* calls (GH-741)
https://github.com/python/cpython/commit/d7fa6b259e00fca04dbf816bfcf4115fdda14bb7


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29859] Return code of pthread_* in thread_pthread.h is not used for perror

[INADA Naoki]

INADA Naoki added the comment:


New changeset fe30339534c602af1123e1402e44a1463f91f2e5 by INADA Naoki in branch 
'3.6':
bpo-29859: Fix error messages from return codes for pthread_* calls (GH-753)
https://github.com/python/cpython/commit/fe30339534c602af1123e1402e44a1463f91f2e5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29859] Return code of pthread_* in thread_pthread.h is not used for perror

[INADA Naoki]

Changes by INADA Naoki :


--
pull_requests: +666

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29859] Return code of pthread_* in thread_pthread.h is not used for perror

[Daniel Birnstiel]

Changes by Daniel Birnstiel :


--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29859] Return code of pthread_* in thread_pthread.h is not used for perror

[INADA Naoki]

INADA Naoki added the comment:

OK, perror() writes to stderr too.  fair enough.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29859] Return code of pthread_* in thread_pthread.h is not used for perror

[Daniel Birnstiel]

Daniel Birnstiel added the comment:

While I agree, fprintf it not really nice, I looked through other parts of the 
python source where information is printed to stderr and fprintf was used there 
as well, so I fell back to it myself.

% grep -rnw . -e "fprintf(stderr," | wc -l  

   178

Using threading and multiprocessing is insane, I know that. Nevertheless the 
error codes returned by the pthread_* calls are processed incorrectly, so I 
would consider this a bug worth fixing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29859] Return code of pthread_* in thread_pthread.h is not used for perror

[INADA Naoki]

INADA Naoki added the comment:

I don't know your patch is worth enough or not. (I dislike fprintf(stderr, ...) 
at least).

But my advice is stop mixing multithreading and fork (except fork+exec).
It's almost impossible.

While Python has GIL, some extension can release GIL and run any C code.
But very vary functions are not fork-safe.  Even malloc and printf are unsafe.

For more information, see "rational" section in 
http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_atfork.html

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29859] Return code of pthread_* in thread_pthread.h is not used for perror

[Daniel Birnstiel]

Daniel Birnstiel added the comment:

While you might scold me for the following code, it is just some fiddling with 
the cpython functions from python side.

Backstory is issue 6721 or related to it. I am working on a multi-process 
service which uses multiprocessing (or rather the billiard fork). While forking 
I run into deadlocks as described in the issue when I fork while the io lock is 
acquired.

In order to find a solution I experimented with the cpython module and tried to 
release the lock manually after a fork. The code can be found attached (tested 
with python 3.6/3.7 on OSX Sierra). While it does not really do what it is 
supposed to do, it shows the problem described in this issue (#29859):

If the internal calls to the pthread_* functions fail, I will only get a 
generic error message:

pthread_mutex_lock[3]: Undefined error: 0
pthread_cond_signal: Undefined error: 0
pthread_mutex_unlock[3]: Undefined error: 0

In reality the produced error messages should be:

pthread_mutex_lock[3]: Invalid argument
pthread_cond_signal: Invalid argument
pthread_mutex_unlock[3]: Invalid argument

This happens because the pthread_* functions do not set the erno variable as 
described in the issue description.

Please note that the issue is not only related to my code, but might affect all 
code using locks. While I suppose there shouldn't be any errors when calling 
the low level functions, the attached patch/pr will make debugging a lot easier 
when it actually happens by displaying the correct error message.

--
Added file: http://bugs.python.org/file46749/test.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29859] Return code of pthread_* in thread_pthread.h is not used for perror

[INADA Naoki]

INADA Naoki added the comment:

Could you give us minimum sample Python code to reproduce the error?

--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29859] Return code of pthread_* in thread_pthread.h is not used for perror

[Daniel Birnstiel]

Changes by Daniel Birnstiel :


--
pull_requests: +654

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29859] Return code of pthread_* in thread_pthread.h is not used for perror

[Daniel Birnstiel]

Daniel Birnstiel added the comment:

I have attached a diff adding a new macro for handling pthread_* status codes. 
Will submit PR as soon as my CLA is approved.

--
keywords: +patch
versions: +Python 3.7
Added file: http://bugs.python.org/file46746/patch.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29859] Return code of pthread_* in thread_pthread.h is not used for perror

[Daniel Birnstiel]

New submission from Daniel Birnstiel:

Python/thread_pthread.h:145 defines the CHECK_STATUS macro used for printing 
error messages in case any of the calls fail.

CHECK_STATUS uses perror for formatting an error message, which relies on the 
global erno being set (see man perror). Since the pthread functions return 
their status code instead of setting erno (which might not even work in 
threaded environments), no additional information is displayed. See for example 
produced by PyThread_release_lock:

pthread_mutex_lock[3]: Undefined error: 0
pthread_cond_signal: Undefined error: 0
pthread_mutex_unlock[3]: Undefined error: 0

The correct solution would be to use strerror(status) in order to show the 
proper message.

--
components: Interpreter Core
messages: 289884
nosy: Birne94
priority: normal
severity: normal
status: open
title: Return code of pthread_* in thread_pthread.h is not used for perror
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com