areusch edited a comment on pull request #6953: URL: https://github.com/apache/tvm/pull/6953#issuecomment-747679635
that is correct. this problem is not specific to sockets--it is just the only place we see it now. it may appear in other contexts, such as in cases where a GPU driver hangs and you cannot ctrl+c it. it is better to solve this broadly using the EINTR hook rather than by requiring all languages to implement sockets on the frontend. I do agree that is an approach, but it just solves the one case rather than this issue more generally. @rkimball is right in saying that Python handles retrying internally--here is how it does it: when you call e.g. `os.read`, an internal CPython function (written in C) enters a loop. Each time, it calls read, and if it gets back `EINTR`, it calls `PyErr_CheckSignals` in a normal (I.e. not signal handler) context. It may get `EINTR` for many reasons, but one could be that a signal handler was invoked. When this happens and CPython is listening for said signal handler, it will have registered a C signal handler by calling `signal`. The OS now aborts `read` and calls this C signal handler. Notably, inside the C signal handler, the Python interpreter can't be resumed to handle the signal in Python, because the signal handler could be invoked at any time and that would be a multithreading catastrophe. So instead CPython sets an internal "signal handler pending" flag. Now two things could happen depending on the value of `siginterrupt`. If `siginterrupt` is set, then the syscall will return `EINTR`. If not, the OS will resume the syscall internally as if nothing happened. In the `EINTR` case, C functions called from Python should call `PyErr_CheckSignals`. Though it is called from a "c function call instruction," `PyErr_CheckSignals` checks if there are any pending Python-side signal handlers and jumps the Python interpreter to those Python-side signal handlers. If one of those raises an exception, the signal handler terminates and `PyErr_CheckSignals` returns 1. It is a bit of a hack to call this function from cython/ctypes, but it worked in my prototype. Note that in the case that `siginterrupt` is 0, a C extension probably should call `PyErr_CheckSignals` if it is not merely wrapping an OS syscall, because if `SIGINT` happened and was swallowed by `siginterrupt`, the C extension does need to handle the side effect of the syscall (i.e. the data returned from os.read should go somewhere), but it should not issue *another* syscall which may block and make Ctrl+C appear to do nothing. I believe this is why `siginterrupt` is automatically set when you set a custom Python signal handler. from Python's perspective, calling TVM's C functions through cython or ctypes is *the same* as calling one of its internal syscall wrapper functions. Python is expecting us to also handle `EINTR` ourselves. so if we add this hook, we are holding up our end of the contract, and solving this not just for socket ops, but providing an easy way to handle this for e.g. GPU syscalls. Finally, new language frontends need only implement this hook, rather than solve this in each separate e.g. socket, GPU, etc case. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
