[issue38104] File descriptor error when subprocess call is used with event loop enabled in main thread

[Andrew Svetlov]

Andrew Svetlov  added the comment:

You spawn too many subprocesses that finish virtually at the same time.
It leads to wakeup_fd overrun.

Python 3.6 is in security mode, sorry (and the fix is impossible).
Python 3.7 has warn_on_full_buffer=False flag for  
https://docs.python.org/3/library/signal.html#signal.set_wakeup_fd but asyncio 
doesn't use it for reasons.

Python 3.8 doesn't subscribe for SIGCHLD by default, thus the provided example 
finishes without any warning.

I'm going to closing the issue as won't fix.

--
resolution:  -> wont fix
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38104] File descriptor error when subprocess call is used with event loop enabled in main thread

[Manoj C]

New submission from Manoj C :

I have been using asyncio to run subprocess calls in a separate thread. For 
this purpose I start an event loop in my main thread as per the recommendation 
- 
https://docs.python.org/3/library/asyncio-subprocess.html#subprocess-and-threads
 .

Now when I use a normal subprocess call in the main thread, I start getting 
following file descriptor error after few iterations:

```
Exception ignored when trying to write to the signal wakeup fd:
BlockingIOError: [Errno 11] Resource temporarily unavailable
```

I have reproduced the problem in a small script and am seeing that the error 
goes away if I do not start the even loop in the main thread.

```
import asyncio
import subprocess
import time


def try_error():
for i in range(1,500):
print(i)
try:
subprocess.run(["ls", "-l"], check=True, stdout=subprocess.PIPE, 
stderr=subprocess.PIPE)
except subprocess.CalledProcessError as e:
print(f"Exception raised {e.stderr}\nOutput {e.stdout}")



def definite_error():
w = asyncio.get_child_watcher()
l = asyncio.get_event_loop()
try_error()


if __name__ == "__main__":
definite_error()
```

This is the smallest subset of the code which can reproduce the error. In the 
original code, I run a asyncio.create_subprocess_exec in a parallel thread. The 
normal subprocess call is part of third party code which call from the main 
thread and hence cannot modify it.

--
components: asyncio
messages: 351795
nosy: Manoj C, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: File descriptor error when subprocess call is used with event loop 
enabled in main thread
type: crash
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com