[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset 3058b7d85697f95573fa042d6b9e4d6e2a9e739c by Pablo Galindo in 
branch 'master':
bpo-33613: Fix test_semaphore_tracker signal tests when using -Werror (GH-9778)
https://github.com/python/cpython/commit/3058b7d85697f95573fa042d6b9e4d6e2a9e739c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +9164
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

After this change tests are failed when ran with -Werror.

$ ./python -Werror -m test -vuall -m 'test_semaphore_tracker_sig*' 
test_multiprocessing_fork test_multiprocessing_forkserver 
test_multiprocessing_spawn
...
==
ERROR: test_semaphore_tracker_sigkill 
(test.test_multiprocessing_fork.TestSemaphoreTracker)
--
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/test/_test_multiprocessing.py", line 4562, 
in test_semaphore_tracker_sigkill
self.check_semaphore_tracker_death(signal.SIGKILL, True)
  File "/home/serhiy/py/cpython/Lib/test/_test_multiprocessing.py", line 4526, 
in check_semaphore_tracker_death
_semaphore_tracker.ensure_running()
  File "/home/serhiy/py/cpython/Lib/multiprocessing/semaphore_tracker.py", line 
64, in ensure_running
warnings.warn('semaphore_tracker: process died unexpectedly, '
UserWarning: semaphore_tracker: process died unexpectedly, relaunching.  Some 
semaphores might leak.

--
...
==
ERROR: test_semaphore_tracker_sigint 
(test.test_multiprocessing_forkserver.TestSemaphoreTracker)
--
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/test/_test_multiprocessing.py", line 4554, 
in test_semaphore_tracker_sigint
self.check_semaphore_tracker_death(signal.SIGINT, False)
  File "/home/serhiy/py/cpython/Lib/test/_test_multiprocessing.py", line 4526, 
in check_semaphore_tracker_death
_semaphore_tracker.ensure_running()
  File "/home/serhiy/py/cpython/Lib/multiprocessing/semaphore_tracker.py", line 
64, in ensure_running
warnings.warn('semaphore_tracker: process died unexpectedly, '
UserWarning: semaphore_tracker: process died unexpectedly, relaunching.  Some 
semaphores might leak.

==
ERROR: test_semaphore_tracker_sigkill 
(test.test_multiprocessing_forkserver.TestSemaphoreTracker)
--
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/test/_test_multiprocessing.py", line 4562, 
in test_semaphore_tracker_sigkill
self.check_semaphore_tracker_death(signal.SIGKILL, True)
  File "/home/serhiy/py/cpython/Lib/test/_test_multiprocessing.py", line 4534, 
in check_semaphore_tracker_death
sem = ctx.Semaphore()
  File "/home/serhiy/py/cpython/Lib/multiprocessing/context.py", line 82, in 
Semaphore
return Semaphore(value, ctx=self.get_context())
  File "/home/serhiy/py/cpython/Lib/multiprocessing/synchronize.py", line 126, 
in __init__
SemLock.__init__(self, SEMAPHORE, value, SEM_VALUE_MAX, ctx=ctx)
  File "/home/serhiy/py/cpython/Lib/multiprocessing/synchronize.py", line 80, 
in __init__
register(self._semlock.name)
  File "/home/serhiy/py/cpython/Lib/multiprocessing/semaphore_tracker.py", line 
104, in register
self._send('REGISTER', name)
  File "/home/serhiy/py/cpython/Lib/multiprocessing/semaphore_tracker.py", line 
111, in _send
self.ensure_running()
  File "/home/serhiy/py/cpython/Lib/multiprocessing/semaphore_tracker.py", line 
64, in ensure_running
warnings.warn('semaphore_tracker: process died unexpectedly, '
UserWarning: semaphore_tracker: process died unexpectedly, relaunching.  Some 
semaphores might leak.

--
...
==
ERROR: test_semaphore_tracker_sigint 
(test.test_multiprocessing_spawn.TestSemaphoreTracker)
--
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/test/_test_multiprocessing.py", line 4554, 
in test_semaphore_tracker_sigint
self.check_semaphore_tracker_death(signal.SIGINT, False)
  File "/home/serhiy/py/cpython/Lib/test/_test_multiprocessing.py", line 4526, 
in check_semaphore_tracker_death
_semaphore_tracker.ensure_running()
  File "/home/serhiy/py/cpython/Lib/multiprocessing/semaphore_tracker.py", line 
64, in ensure_running
warnings.warn('semaphore_tracker: process died unexpectedly, '
UserWarning: semaphore_tracker: process died unexpectedly, relaunching.  Some 
semaphores might leak.

==
ERROR: test_semaphore_tracker_sigkill 
(test.test_multiprocessing_spawn.TestSemaphoreTracker)
--
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/test/_test_multiprocessing.py", line 4562, 
in test_semaphore_tracker_sigkill

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

Change by STINNER Victor :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

STINNER Victor  added the comment:

> Which bug? It's only a failing test.

Linux vendors run the full test suite. If the test suite fails, the build of 
the package fails as well. It's annoying.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I don't want to spend hours arguing.  This issue is sufficiently rare and 
unlikely in normal conditions that I don't think we should risk regressions by 
trying to fix it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

STINNER Victor  added the comment:

By the way, "no user-visible bug is fixed here": I agree that it's not easy to 
trigger manually the bug (when pressing CTRL+C), but I don't see why an user 
couldn't hit this bug. The race condition is now obvious to me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Le 04/09/2018 à 11:16, STINNER Victor a écrit :
> 
>> They work fine otherwise.
> 
> The race condition impacts everyone. It's just less likely if your computer 
> is fast enough. The bug is much more likely if you "force" the bad path:

Which bug? It's only a failing test.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

STINNER Victor  added the comment:

> Please only skip those tests on buildbots.

There is no easy wait to only skip a test on buildbots.

> They work fine otherwise.

The race condition impacts everyone. It's just less likely if your computer is 
fast enough. The bug is much more likely if you "force" the bad path:

diff --git a/Lib/multiprocessing/semaphore_tracker.py 
b/Lib/multiprocessing/semaphore_tracker.py
index 3b50a46ddc..7261b43725 100644
--- a/Lib/multiprocessing/semaphore_tracker.py
+++ b/Lib/multiprocessing/semaphore_tracker.py
@@ -107,6 +107,8 @@ getfd = _semaphore_tracker.getfd
 def main(fd):
 '''Run semaphore tracker.'''
 # protect the process from ^C and "killall python" etc
+import time
+time.sleep(0.5)
 signal.signal(signal.SIGINT, signal.SIG_IGN)
 signal.signal(signal.SIGTERM, signal.SIG_IGN)
 
diff --git a/Lib/test/_test_multiprocessing.py 
b/Lib/test/_test_multiprocessing.py
index 5c625dd495..6f9f7583e2 100644
--- a/Lib/test/_test_multiprocessing.py
+++ b/Lib/test/_test_multiprocessing.py
@@ -4193,7 +4193,6 @@ class TestSemaphoreTracker(unittest.TestCase):
 _multiprocessing.sem_unlink(name1)
 p.terminate()
 p.wait()
-time.sleep(2.0)
 with self.assertRaises(OSError) as ctx:
 _multiprocessing.sem_unlink(name2)
 # docs say it should be ENOENT, but OSX seems to give EINVAL

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Please only skip those tests on buildbots. They work fine otherwise.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

STINNER Victor  added the comment:

> If buildbots hurt, I suggest skipping the tests on the buildbots.

Let's do that. Pablo: do you want to write a PR to always skip 
TestSemaphoreTracker with a reference to this issue? Example: skipIf(True, 
"bpo-33613: the test has a race condition").

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Antoine Pitrou]

Antoine Pitrou  added the comment:

As I wrote on Github: no user-visible bug is fixed here, and we shouldn't risk 
introducing regressions by backporting those changes.

If buildbots hurt, I suggest skipping the tests on the buildbots.

--
versions:  -Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

STINNER Victor  added the comment:

IMHO the fix should be backported to Python 3.6 and 3.7.

It cannot be backported to 2.7 since 2.7 doesn't have signal.pthread_sigmask().

--
versions: +Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[miss-islington]

Change by miss-islington :


--
pull_requests: +8515

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset ec74d187f50a8a48f94eb37023300583fbd644cc by Antoine Pitrou (Pablo 
Galindo) in branch 'master':
bpo-33613, test_semaphore_tracker_sigint: fix race condition (#7850)
https://github.com/python/cpython/commit/ec74d187f50a8a48f94eb37023300583fbd644cc


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +8018

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-31687.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I have modified the PR to avoid creating an extra pipe in the tracker (the pipe 
is created as part of the test). To allow testing with the -R option:

./python -m test test_multiprocessing_fork -v -m test_semaphore_tracker_sigint 
-R 3:3

I had to assure that we restart the tracker in the test so it does not reuse a 
closed file descriptor.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

The process seems to use its own stderr and it cannot be captured from the 
parent without modifying it AFAIK.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Is it possible to notify the ping / noop reply on stderr instead of introducing 
a new pipe?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I have updated the PR implementing a PONG command in the semaphore and updating 
the test. I have tested on the failing buildbots (gcc110.fsffrance.org) and it 
works. Notice that a new pipe for the child to write was needed as the child is 
continuously reading from the original one and therefore we cannot use the same 
one.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Tal Einat]

Tal Einat  added the comment:

For the "no-op" we'll need a way to know that it was processed.  To that end, 
I'd make the semaphore tracker recognize a new command e.g. "PING", upon which 
it would write something e.g. "PONG" into its end of the pipe.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

@pitrou I think that should be ideal. The reason is that if we send an invalid 
command the cache is not cleared and the semaphore tracker will warn when the 
process finishes which is precisely what the test is avoiding.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Additionally, as I commented in the PR, apart from the race condition I think 
the test should be modified so it fails if the warning is raised and not 
expected:

>The problem is that the test was silently failing when SIGINT was >being 
>delivered as the test does not check that no warnings are >raised. Notice that 
>the presence of a warning means that the process >died and therefore SIGINT 
>did kill the process, which is precisely >what the test checks that does not 
>happen.

>I think in order to check that the test works as intended without the >need to 
>run the suite with -Wall, we need to modify the test.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Sorry, I was wrong, it does not die. I am investigating this solution using 
stderr.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Antoine Pitrou]

Antoine Pitrou  added the comment:

We can easily add a 'NOOP' command to the semaphore tracker if it helps solve 
that issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

@taleinat Sadly, if the semaphore_tracker receives an invalid command, it 
raises and dies, making the test impossible:

 cmd, name = line.strip().split(b':')
 if cmd == b'REGISTER':
 cache.add(name)
 elif cmd == b'UNREGISTER':
 cache.remove(name)
 else:
 raise RuntimeError('unrecognized command %r' % cmd)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Tal Einat]

Tal Einat  added the comment:

You can know that the semaphore tracker is running by sending it an invalid 
command and seeing the error written to stderr.  The semaphore tracker either 
inherits sys.stderr from the parent process (if possible) or uses its own.  
This is a bit of a hack, but might be worth it to avoid another wasted second 
on every run of the test suite.

--
nosy: +taleinat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +7461
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

There are two problems in test_semaphore_tracker_sigint. One of the problems is 
that if `should_die` is False, the code does not check
that no warning is raised (and the warning is being raised, so this
the error has pass silently.) The other problem is a race condition
between the parent calls `_semaphore_tracker.ensure_running()` (which in turn 
spawns the semaphore_tracker using `_posixsubprocess.fork_exec`) and the child 
registering the signals. So basically, the parent sends the signal before the 
child protects against the signal. Modifying:

_semaphore_tracker.ensure_running()
pid = _semaphore_tracker._pid
os.kill(pid, signum)
time.sleep(1.0)  # give it time to die

to

_semaphore_tracker.ensure_running()
pid = _semaphore_tracker._pid
time.sleep(1.0) # Give time for the child to register the signal 
handlers
os.kill(pid, signum)
time.sleep(1.0)  # give it time to die

fix the issue. I have tested this on one of the most grumpy and slow buildbots 
(gcc110) and it works.

I cannot think of a different way of waiting for the child to register the 
signals without modifying the child code for testing so I am not sure that we 
can do anything rather than the sleep.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

STINNER Victor  added the comment:

> I would like to work on this issue and on issue31463

Say it on issue31463 as well on that case ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I would like to work on this issue and on issue31463

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I would like to work on this issue and on bpo31463

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
Removed message: https://bugs.python.org/msg320061

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
assignee:  -> pablogsal
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

See also some information about this in issue31463.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

STINNER Victor  added the comment:

(Oops, I attached a file and added two comments to this issue, whereas I wanted 
to comment bpo-33612.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

Change by STINNER Victor :


Removed file: https://bugs.python.org/file47611/bug.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

Change by STINNER Victor :


--
Removed message: https://bugs.python.org/msg317376

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

Change by STINNER Victor :


--
Removed message: https://bugs.python.org/msg317375

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

STINNER Victor  added the comment:

Using bug.py, the assertion fails at:

python: Python/pystate.c:589: PyThreadState_Clear: Assertion 
`tstate->exc_info->previous_item == NULL' failed.
Aborted (core dumped)
(...)
vstinner@apu$ gdb ./python -c coredump-python.24130
GNU gdb (GDB) Fedora 8.0.1-36.fc27
(..)
Core was generated by `./python bug.py'.
Program terminated with signal SIGABRT, Aborted.

#0  0x7f5136075660 in raise () from /lib64/libc.so.6
[Current thread is 1 (Thread 0x7f5136f64040 (LWP 24130))]

(gdb) where
#0  0x7f5136075660 in raise () from /lib64/libc.so.6
#1  0x7f5136076c41 in abort () from /lib64/libc.so.6
#2  0x7f513606df7a in __assert_fail_base () from /lib64/libc.so.6
#3  0x7f513606dff2 in __assert_fail () from /lib64/libc.so.6
#4  0x00591dde in PyThreadState_Clear (tstate=0x1b7a430) at 
Python/pystate.c:589
#5  0x00590274 in PyInterpreterState_Clear (interp=0x1b23930) at 
Python/pystate.c:204
#6  0x0058cf4c in Py_FinalizeEx () at Python/pylifecycle.c:1153
#7  0x004257af in pymain_main (pymain=0x7ffe5f82f200) at 
Modules/main.c:2664
#8  0x00425915 in _Py_UnixMain (argc=2, argv=0x7ffe5f82f448) at 
Modules/main.c:2697
#9  0x0041f1e7 in main (argc=2, argv=0x7ffe5f82f448) at 
./Programs/python.c:15

(gdb) info threads
  Id   Target Id Frame 
  1Thread 0x7f5136f64040 (LWP 24130) 0x7f5136075660 in raise () from 
/lib64/libc.so.6
* 2Thread 0x7f512e8c6700 (LWP 24138) 0x7f5136b56be6 in 
pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  3Thread 0x7f512e0c5700 (LWP 24139) 0x7f5136b56be6 in 
pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  4Thread 0x7f512d8c4700 (LWP 24147) 0x7f5136b56be6 in 
pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  5Thread 0x7f512f0c7700 (LWP 24137) 0x7f5136b56be6 in 
pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0

(gdb) t 2
[Switching to thread 2 (Thread 0x7f512e8c6700 (LWP 24138))]
#0  0x7f5136b56be6 in pthread_cond_timedwait@@GLIBC_2.3.2 () from 
/lib64/libpthread.so.0

(gdb) where
#0  0x7f5136b56be6 in pthread_cond_timedwait@@GLIBC_2.3.2 () from 
/lib64/libpthread.so.0
#1  0x0053458a in PyCOND_TIMEDWAIT (cond=0xa07248 <_PyRuntime+1224>, 
mut=0xa07278 <_PyRuntime+1272>, us=5000) at Python/condvar.h:90
#2  0x005349a3 in take_gil (tstate=0x1b87980) at Python/ceval_gil.h:208

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

STINNER Victor  added the comment:

Attached bug.py reproduces the bug with a daemonic thread but without fork():

$ ./python bug.py 
ls coredpython: Python/pystate.c:589: PyThreadState_Clear: Assertion 
`tstate->exc_info->previous_item == NULL' failed.
Aborted (core dumped)

(You might have to run the script a few times to get the crash, it's not 
deterministic.)

--
Added file: https://bugs.python.org/file47611/bug.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

STINNER Victor  added the comment:

Note: I found this bug while working on bpo-33612.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

New submission from STINNER Victor :

test_semaphore_tracker_sigint() emits a warning. If the test is run with -W 
error, the test fails.

vstinner@apu$ ./python -m test test_multiprocessing_fork -v -m 
test_semaphore_tracker_sigint
(...)
test_semaphore_tracker_sigint 
(test.test_multiprocessing_fork.TestSemaphoreTracker) ... 
/home/vstinner/prog/python/master/Lib/multiprocessing/semaphore_tracker.py:55: 
UserWarning: semaphore_tracker: process died unexpectedly, relaunching.  Some 
semaphores might leak.
  warnings.warn('semaphore_tracker: process died unexpectedly, '
ok
(...)
Tests result: SUCCESS

vstinner@apu$ ./python  -Werror  -m test test_multiprocessing_fork -v -m 
test_semaphore_tracker_sigint
(...)
==
ERROR: test_semaphore_tracker_sigint 
(test.test_multiprocessing_fork.TestSemaphoreTracker)
--
Traceback (most recent call last):
  File "/home/vstinner/prog/python/master/Lib/test/_test_multiprocessing.py", 
line 4533, in test_semaphore_tracker_sigint
self.check_semaphore_tracker_death(signal.SIGINT, False)
  File "/home/vstinner/prog/python/master/Lib/test/_test_multiprocessing.py", 
line 4521, in check_semaphore_tracker_death
sem = ctx.Semaphore()
  File "/home/vstinner/prog/python/master/Lib/multiprocessing/context.py", line 
82, in Semaphore
return Semaphore(value, ctx=self.get_context())
  File "/home/vstinner/prog/python/master/Lib/multiprocessing/synchronize.py", 
line 127, in __init__
SemLock.__init__(self, SEMAPHORE, value, SEM_VALUE_MAX, ctx=ctx)
  File "/home/vstinner/prog/python/master/Lib/multiprocessing/synchronize.py", 
line 81, in __init__
register(self._semlock.name)
  File 
"/home/vstinner/prog/python/master/Lib/multiprocessing/semaphore_tracker.py", 
line 83, in register
self._send('REGISTER', name)
  File 
"/home/vstinner/prog/python/master/Lib/multiprocessing/semaphore_tracker.py", 
line 90, in _send
self.ensure_running()
  File 
"/home/vstinner/prog/python/master/Lib/multiprocessing/semaphore_tracker.py", 
line 55, in ensure_running
warnings.warn('semaphore_tracker: process died unexpectedly, '
UserWarning: semaphore_tracker: process died unexpectedly, relaunching.  Some 
semaphores might leak.
(...)
Tests result: FAILURE

--
components: Tests
messages: 317372
nosy: davin, pitrou, vstinner
priority: normal
severity: normal
status: open
title: test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W 
error
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com