> On 29. Jan 2018, at 21:52, Holger Freyther <hol...@freyther.de> wrote:
> 

Hey,

I wish I had better news. Instead of an end to end test I realize that picking 
"asyncio" was a grave mistake. Besides the lack AF_UNIX SOCK_DGRAM support, the 
process support is dangerous and doesn't scale. Today I stumbled into SIGCHLD 
handling of spawned processes. While normally neither virthphy/mobile would 
prematurely exit, a few exits could take the entire test down.

Speculating from the behavior of strace:

* SIGCHLD arrives
* Something will be written into one end of a socketpair[1]
* In the python code on wait(2) will be called on every registered process
(https://github.com/python/cpython/blob/3.6/Lib/asyncio/unix_events.py#L819)

As more processes exit the main python code is still in the for loop and the 
buffer runs full. It also means that the main application will be busy 
executing Xk syscalls instead of launching processes or events.


It is unfortunate that I only notice now but probably still better than having 
to deal mysterious failures of processes not starting and all tests failing. As 
not many people here know Go. I think I will stay with python but use the lower 
level epoll API directly. So we end up with a single "select" system... I 
should finally have something during the week.

I hope all of you had a nice fosdem!

holger


[1] A common trick to notify one thread or part of the application of an event. 
There is not much that is allowed in a signal handler so using a buffer is a 
reasonable approach.. It should forward the PID of the process that exited 
though.

Reply via email to