otterc commented on issue #27240: [SPARK-30512] Added a dedicated boss event loop group URL: https://github.com/apache/spark/pull/27240#issuecomment-577863837 @tgravescs The boss thread is just responsible for accepting the connection and associating a worker thread with the accepted channel. Once a channel is registered with a worker which is done by the boss thread, then the worker thread starts reading from the channel and processing the messages and then writes back the response. The actual processing of SASL authentication messages will be handled by worker threads not the boss thread. Currently in External Shuffle Service, these are the 2 event loop groups: 1. Server IO event loop group. This is the worker group as well as the boss group. 2. Chunk-fetch handler event loop group. Since the worker group and the boss group is same, one of the worker thread is also the boss thread. In a busy shuffle service, the boss thread (which is also a worker thread) is busy with the channels associated with it. These are 2 main things the Server IO thread is doing: 1. Reading from channel 2. Running all the pending tasks in its queue. Most of these tasks are `WriteAndFlushTasks` which are added by the Chunk-fetch threads. Since its busy with these tasks, there is delay to accept new connections and register the channel. SPARK-24355 is still needed because it delegates processing of chunk-fetch requests to chunk-fetch threads. The processing of ChunkFetchRequests is blocked on disk I/O mostly due to disk contentions from the random read operations initiated by all the ChunkFetchRequests. If we don't delegate this to another event loop group, then worker threads will be blocked and processing of lightweight RPC like SASL will still timed-out.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
