[issue33329] sigaddset() can fail on some signal numbers

[Ned Deily]

Change by Ned Deily :


--
Removed message: https://bugs.python.org/msg342090

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Ned Deily]

Ned Deily  added the comment:


New changeset 2226139aa2b69047cb54dbcfd79f5c2e36f98653 by larryhastings (Cheryl 
Sabella) in branch '3.4':
[3.4] bpo-33329: Fix multiprocessing regression on newer glibcs (GH-6575) 
(#12145)
https://github.com/python/cpython/commit/2226139aa2b69047cb54dbcfd79f5c2e36f98653


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Larry Hastings]

Larry Hastings  added the comment:

Now fixed in 3.4 and 3.5.  I can cut the RCs.  Huzzah!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Larry Hastings]

Larry Hastings  added the comment:


New changeset 2226139aa2b69047cb54dbcfd79f5c2e36f98653 by larryhastings (Cheryl 
Sabella) in branch '3.4':
[3.4] bpo-33329: Fix multiprocessing regression on newer glibcs (GH-6575) 
(#12145)
https://github.com/python/cpython/commit/2226139aa2b69047cb54dbcfd79f5c2e36f98653


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Larry Hastings]

Larry Hastings  added the comment:


New changeset 8ec1fd11f2d524859cfefae76458fcfd22decf65 by larryhastings (Cheryl 
Sabella) in branch '3.5':
[3.5] bpo-33329: Fix multiprocessing regression on newer glibcs (GH-6575) 
(#12144)
https://github.com/python/cpython/commit/8ec1fd11f2d524859cfefae76458fcfd22decf65


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Cheryl Sabella]

Change by Cheryl Sabella :


--
pull_requests: +12145

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Cheryl Sabella]

Change by Cheryl Sabella :


--
pull_requests: +12144
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Miro Hrončok]

Miro Hrončok  added the comment:

I am not able to do PRs right now but here are the Fedora patches:

https://src.fedoraproject.org/rpms/python34/blob/master/f/00302-fix-multiprocessing-regression-on-newer-glibcs.patch
https://src.fedoraproject.org/rpms/python35/blob/master/f/00302-fix-multiprocessing-regression-on-newer-glibcs.patch


They seem quite identical to the applied 3.6 patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Larry Hastings]

Larry Hastings  added the comment:

These issues also occur on Python 3.4 and 3.5.  And I'm now upgraded to Ubuntu 
18.10 which I guess has the new version of glibc. The regression test suite for 
both 3.4 and 3.5 blocks forever on three tests (multiprocessing_fork, 
multiprocessing_forkserver, and multiprocessing_spawn).  This affects my 
ability as RM to run the regression test suite, not to mention affecting the 
fundamental usability and trustability of 3.4 and 3.5 releases.


Can I please get backports of this patch to 3.4 and 3.5?  Note that I consider 
this a release blocker for 3.4 and 3.5.  And I'd expected to tag 3.4.10rc1 and 
3.5.7rc1 in about twenty-four hours.  So I sure would appreciate a quick 
turnaround on this if folks are available--otherwise I'll have to slip the 
schedule.  Sorry for the late notice!

--
nosy: +larry, lukasz.langa
resolution: fixed -> 
stage: resolved -> needs patch
status: closed -> open
versions: +Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Miro Hrončok]

Miro Hrončok  added the comment:

Antoine, thank you very much for swift communication and fix.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Miro, this should be fixed now. Please reopen if it isn't.

The sigfillset() functionality will be exposed to Python users in bpo-2.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset b0ca398cabd2d2ea2d66fa50b08e297a60388c75 by Antoine Pitrou in 
branch '3.6':
[3.6] bpo-33329: Fix multiprocessing regression on newer glibcs (GH-6575) 
(GH-6582)
https://github.com/python/cpython/commit/b0ca398cabd2d2ea2d66fa50b08e297a60388c75


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Change by Antoine Pitrou :


--
pull_requests: +6282

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Change by Antoine Pitrou :


--
nosy: +njs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset 75a3e3d5bc0be1ce41289b661e7c53039cf3d5ba by Antoine Pitrou (Miss 
Islington (bot)) in branch '3.7':
bpo-33329: Fix multiprocessing regression on newer glibcs (GH-6575) (GH-6579)
https://github.com/python/cpython/commit/75a3e3d5bc0be1ce41289b661e7c53039cf3d5ba


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[miss-islington]

Change by miss-islington :


--
pull_requests: +6280

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[miss-islington]

Change by miss-islington :


--
pull_requests: +6279

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset 25038ecfb665bef641abf8cb61afff7505b0e008 by Antoine Pitrou in 
branch 'master':
bpo-33329: Fix multiprocessing regression on newer glibcs (GH-6575)
https://github.com/python/cpython/commit/25038ecfb665bef641abf8cb61afff7505b0e008


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Change by Antoine Pitrou :


--
nosy: +neologix

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Adhemerval Zanella]

Adhemerval Zanella  added the comment:

What I mean is for master glibc, which contains BZ#22391, 'sigaddset' will fail 
do add signal 32 and 33.  But since you are now ignoring 'sigaddset' return 
code, it does not matter.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Not sure what you mean.  In the example above, I try to block signals 32 and 33 
using pthread_sigmask(), but the pthread_sigmask() return value shows they 
weren't blocked, which is ok.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Adhemerval Zanella]

Adhemerval Zanella  added the comment:

Yes, this is the issue I referred in previous comment [1]. Unfortunately it is 
only fixed on master (which will become 2.28).

[1] https://sourceware.org/bugzilla/show_bug.cgi?id=22391

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Change by Antoine Pitrou :


--
keywords: +patch
pull_requests: +6276
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Right, but it seems that even if sigaddset() allowed you to "set" signals 32 
and 33, that would be ignored by pthread_sigmask().

This is what I get here (Ubuntu 16.04, glibc 2.23):

>>> signal.pthread_sigmask(signal.SIG_BLOCK, range(1, 65))
set()
>>> signal.pthread_sigmask(signal.SIG_BLOCK, range(1, 65))
{, , , 
, , , 
, , , 
, , , 
, , 16, , 
, , , 
, , , 
, , , 
, , , 
, , 35, 36, 37, 38, 39, 40, 41, 42, 
43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 
63, }

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Adhemerval Zanella]

Adhemerval Zanella  added the comment:

Yes I am aware, but I can't see a really portable way to provide the same 
functionality as 'sigfillset'.  Ideally a libc implementation would return 
EINVAL on 'sigaddset' for invalid signals, but even for glibc this is not true 
(the very issue I fixed).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I don't think we want to be in the business of maintaining a list of 
cross-platform of supported signal values.  Python may be compiled on bizarre 
non-glibc systems (or even systems with a proprietary libc).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Adhemerval Zanella]

Adhemerval Zanella  added the comment:

One option would be to create a list of possible defined signals and check if 
the signal is on the list.  For realtime signals, it just a matter to check if 
SIGRTMIN <= signal <= SIGRTMAX.

The glibc defined signals can be checked at tst-signal.c [1] or from main 
signal(7).  It should cover usual ISO C, POSIX, and some linux arch-specific 
signals, but you will still need to check if other OS defined extra signals 
uses elsewhere (another option would to add this check only for Linux/glibc).

[1] 
https://sourceware.org/git/?p=glibc.git;a=blob;f=signal/tst-sigset.c;h=a2b764de5ad66ee960c94ec18df75a07fce4b9a6;hb=HEAD

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Change by Antoine Pitrou :


--
assignee:  -> pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> Why not export and use the canonical way of 
> sigemptyset/sigfillset/sigaddset/sigdelset/sigismember instead of pushing for 
> more potential non-conformant code? 

I agree this is the proper fix and that's what I plan to do in Python 3.8.  For 
Python 3.7 and earlier, though, we cannot add new features anymore, which is 
why I'm leaning towards a variant of the patch I showed above (which also 
minimizes the risk for regressions by introducing and using a new API).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Adhemerval Zanella]

Adhemerval Zanella  added the comment:

Also keep in mind that POSIX [1] specifies that sigaddset *may* fail with 
EINVAL for not support signals, so a conforming implementation may not fail on 
sigaddset and still remove the internal signal in a sigprocmask (uclibc for 
instance).

[1] http://pubs.opengroup.org/onlinepubs/9699919799/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Adhemerval Zanella]

Adhemerval Zanella  added the comment:

Why not export and use the canonical way of 
sigemptyset/sigfillset/sigaddset/sigdelset/sigismember instead of pushing for 
more potential non-conformant code? For glibc sigfillset will correctly fill 
all the signal set structure while removing the internal used signals. This is 
par what POSIX specifies [1] where it states that: 

'either sigemptyset() or sigfillset() must be called prior to any other use of 
the signal set'

And more importantly:

'For example, blocking or ignoring an implementation-defined signal may have 
undesirable side-effects, whereas the default action for that signal is 
harmless. In such a case, it would be preferable for such a signal to be 
excluded from the signal set returned by sigfillset().'

Also keep that since is an implementation detail, different libcs can use 
different internal signals. UCLIBC, for instance, uses the same 2 signals as 
GLIBC, however MUSL allocates signal 32, 33, and 34 for internal usage (and 
excludes in sigfillset and warns with EINVAL on sigaddset). 

[1] http://pubs.opengroup.org/onlinepubs/9699919799/

--
nosy: +azanella

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Miro Hrončok]

Miro Hrončok  added the comment:

This indeed makes the tests pass again:

# python3.7 -m test.regrtest test_multiprocessing_fork 
test_multiprocessing_forkserver test_multiprocessing_spawn
Run tests sequentially
0:00:00 load avg: 1.02 [1/3] test_multiprocessing_fork
/usr/lib64/python3.7/multiprocessing/semaphore_tracker.py:55: UserWarning: 
semaphore_tracker: process died unexpectedly, relaunching.  Some semaphores 
might leak.
  warnings.warn('semaphore_tracker: process died unexpectedly, '
0:01:05 load avg: 1.00 [2/3] test_multiprocessing_forkserver -- 
test_multiprocessing_fork passed in 1 min 6 sec
0:02:16 load avg: 1.69 [3/3] test_multiprocessing_spawn -- 
test_multiprocessing_forkserver passed in 1 min 11 sec
test_multiprocessing_spawn passed in 1 min 23 sec
All 3 tests OK.

Total duration: 3 min 40 sec
Tests result: SUCCESS

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Miro, can you give the following patch a try?

diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c
index 7916160..ca76a20 100644
--- a/Modules/signalmodule.c
+++ b/Modules/signalmodule.c
@@ -819,7 +819,6 @@ iterable_to_sigset(PyObject *iterable, sigset_t *mask)
 int result = -1;
 PyObject *iterator, *item;
 long signum;
-int err;
 
 sigemptyset(mask);
 
@@ -841,11 +840,10 @@ iterable_to_sigset(PyObject *iterable, sigset_t *mask)
 Py_DECREF(item);
 if (signum == -1 && PyErr_Occurred())
 goto error;
-if (0 < signum && signum < NSIG)
-err = sigaddset(mask, (int)signum);
-else
-err = 1;
-if (err) {
+if (0 < signum && signum < NSIG) {
+(void) sigaddset(mask, (int)signum);
+}
+else {
 PyErr_Format(PyExc_ValueError,
  "signal number %ld out of range", signum);
 goto error;

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> We could do that, but I'd rather something not multiprocessing-specific

... and also not Linux-specific.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

We could do that, but I'd rather something not multiprocessing-specific.  I 
think we have to ignore the return value of sigaddset() until issue2 allows 
us to be smarter.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Miro Hrončok]

Miro Hrončok  added the comment:

Should the fix be to exclude 32 and 33 from 
multiprocessing.resource_sharer:_serve?

if hasattr(signal, 'pthread_sigmask'):
signal.pthread_sigmask(signal.SIG_BLOCK, range(1, signal.NSIG))

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Setting as release blocker as it impacts functionality of the multiprocessing 
module.

--
nosy: +ned.deily
priority: critical -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Miro Hrončok]

Miro Hrončok  added the comment:

Python 2 testsuite runs fine.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Running the whole test suite sounds fine.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Miro Hrončok]

Miro Hrončok  added the comment:

I cannot find a corresponding test in 2.7. Running the entire build (incl. 
tests) to see what happens.

Is there a command or a couple I could try instead? I'm afraid I don't 
understand how does that test work (I seems a bit complicated)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Miro, could you check whether 2.7 is affected too?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Change by Antoine Pitrou :


--
priority: normal -> critical
stage:  -> needs patch
title: test_multiprocessing_fork, _forkserver, _spawn hangs: ValueError: signal 
number 32 out of range -> sigaddset() can fail on some signal numbers
type:  -> behavior
versions: +Python 3.8 -Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com