lidavidm commented on PR #36986: URL: https://github.com/apache/arrow/pull/36986#issuecomment-1673562741
gRPC themselves do something quite different; I don't know if they ran into challenges like this, but they sidestep this neatly: https://ep2020.europython.eu/media/conference/slides/AfUnDYW-grpc-python-c-extensions-and-asyncio.pdf > - gRPC Asyncio Python application would start a separated thread for polling gRPC events > - This thread won’t use any Python object, during the polling > - Events would be added into a C++ queue > - Asyncio loop will be woken up by writing into a socket (Note that grpcio in Python works quite differently and implements a lot of what gRPC-C++ provides itself, so that may also explain the architecture. In particular, they have to drive the gRPC event loop, while gRPC-C++ manages that for us.) For now, though, I think the easy way would be to have a single-thread thread pool per client that's used to trigger these callbacks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
