[issue26793] uuid causing thread issues when forking using os.fork py3.4+

[Andre Merzky]

Andre Merzky added the comment:

This one might be related:

https://bugs.python.org/issue27889

--
nosy: +Andre Merzky

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26793] uuid causing thread issues when forking using os.fork py3.4+

[Steven Adams]

Steven Adams added the comment:

anyone got any other thoughts on this??

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26793] uuid causing thread issues when forking using os.fork py3.4+

[Chase Sterling]

Chase Sterling added the comment:

@STINNER Victor Your example is starting a thread before calling fork, the 
other examples just init a threading.Thread class before the fork (I imagine 
the OS thread is not created at that point.) Are you saying that just 
instantiating a threading.Thread class before forking (without actually 
starting the thread) is bad?

--
nosy: +Chase Sterling

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26793] uuid causing thread issues when forking using os.fork py3.4+

[Steven Adams]

Steven Adams added the comment:

Ok but the question still remains, why does it only happen on py3.4+??

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26793] uuid causing thread issues when forking using os.fork py3.4+

[Xiang Zhang]

Xiang Zhang added the comment:

I should not make more noise after realizing it's matter of thread and process. 
Sorry. :-( Except this one. :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26793] uuid causing thread issues when forking using os.fork py3.4+

[STINNER Victor]

STINNER Victor added the comment:

Hum, I should elaborate my explanation :-) os.fork() removes all threads except 
of the current thread. It's clearly stated in the fork manual page, but the 
Python os.fork() doesn't say anything about threads.
https://docs.python.org/dev/library/os.html#os.fork

Short example:
---
import os, threading, time

t = threading.Thread(target=time.sleep, args=(3.0,))
t.start()

print("First process", threading.enumerate())

pid = os.fork()
if pid != 0:
os.waitpid(pid, 0)
else:
print("Child process", threading.enumerate())
---

Output:
---
First process [<_MainThread(MainThread, started 140737353955072)>, 
]
Child process [<_MainThread(MainThread, started 140737353955072)>]
---

The thread is removed in the child process.

Again, *don't create threads before fork*. More generally, don't create any 
resource before fork: don't open files, don't create locks, don't open a 
connection to a database, don't start an event loop, etc.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26793] uuid causing thread issues when forking using os.fork py3.4+

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26793] uuid causing thread issues when forking using os.fork py3.4+

[Xiang Zhang]

Xiang Zhang added the comment:

I write a simple example which behaviour is like Steven's example.

import os
import sys
import threading
import ctypes
import time

def test():
while True:
print(time.time())

_thread = threading.Thread(target=test)

pid = os.fork()
if pid > 0:
sys.exit(0)

_thread.start()

Run this with py3.4 emits one print and then comes fatal error:

[code]$ python3 ctypes_test.py 
1460964519.9721818
[code]$ Fatal Python error: could not acquire lock for <_io.BufferedWriter 
name=''> at interpreter shutdown, possibly due to daemon threads

Thread 0x7f41fdadf700 (most recent call first):
  File "ctypes_test.py", line 9 in test
  File "/usr/lib/python3.4/threading.py", line 868 in run
  File "/usr/lib/python3.4/threading.py", line 920 in _bootstrap_inner
  File "/usr/lib/python3.4/threading.py", line 888 in _bootstrap

Current thread 0x7f41ff9a9700 (most recent call first):
^C

Without import ctypes, it's OK.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26793] uuid causing thread issues when forking using os.fork py3.4+

[STINNER Victor]

STINNER Victor added the comment:

IMHO it's a bad idea to create a thread before forking. I suggest you to not 
create any kind of resource before calling daemonize().

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26793] uuid causing thread issues when forking using os.fork py3.4+

[Xiang Zhang]

Xiang Zhang added the comment:

I am not sure what the real problem is. What I can see now it that it seems 
ctypes affects process fork. If you create the thread after fork, for example, 
in wait, your example works well. But if the thread participates in process 
fork, it fails.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26793] uuid causing thread issues when forking using os.fork py3.4+

[Steven Adams]

Steven Adams added the comment:

My workaround didn't work either.. We are a number of third party libs 
including flask. As soon as i import flask the issues remains.. Maybe something 
with flask is importing ctypes?

Something aint right..

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26793] uuid causing thread issues when forking using os.fork py3.4+

[Xiang Zhang]

Xiang Zhang added the comment:

Sorry. I forgot to take that into consideration. I think now the problem 
appears because of ctypes. Simply import ctypes instead of uuid can also lead 
to your problem.

--
nosy: +amaury.forgeotdarc, belopolsky, meador.inge

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26793] uuid causing thread issues when forking using os.fork py3.4+

[Steven Adams]

Steven Adams added the comment:

Shouldn't that mean it also breaks on py3.3?

As a workaround i just import uuid later within the thread method.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26793] uuid causing thread issues when forking using os.fork py3.4+

[Xiang Zhang]

Xiang Zhang added the comment:

It seems a matter of lib uuid. The comments in uuid.py tells that the module is 
not thread-safe. I try to comment out the code using ctypes and rerun your 
sample, it works well. 

In uuid's documentation it does not mention the non-thread-safe characteristic. 
Maybe we should add it.

--
nosy:  -pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26793] uuid causing thread issues when forking using os.fork py3.4+

[Xiang Zhang]

Changes by Xiang Zhang :


--
nosy: +pitrou, xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26793] uuid causing thread issues when forking using os.fork py3.4+

[Steven Adams]

Steven Adams added the comment:

I forgot to mention if i remove import uuid all works as expected.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26793] uuid causing thread issues when forking using os.fork py3.4+

[Steven Adams]

New submission from Steven Adams:

I've ran into a strange issue after trying to port a project to support py 3.x

The app uses a double os.fork to run in the background. On py 3.4+ it seems 
that when you have an import uuid statement it causes threading.threads to 
always return false on is_alive()..

Here is an example of the issue. You can see i've imported uuid. This script 
should fork into the background and stay alive for at least 3 loops (3 seconds) 
but it dies after 1 loop as self._thread.is_alive() return False??

http://paste.pound-python.org/show/WbDkqPqu94zEstHG6Xl1/

This does NOT happen in py 2.7 or py3.3. Only occurs py3.4+

--
components: Interpreter Core
messages: 263640
nosy: Steven Adams
priority: normal
severity: normal
status: open
title: uuid causing thread issues when forking using os.fork py3.4+
versions: Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com