zjf2012 edited a comment on issue #23560: [SPARK-26632][Core] Separate Thread Configurations of Driver and Executor URL: https://github.com/apache/spark/pull/23560#issuecomment-471375430 @attilapiros, thanks for you help. I just updated the description with formatted tables. Almost forgot to respond to your wonder. Besides memory and threads, more threads than necessary mean more context switch and more contention. See below details we responded to our customer they had the same wonder. 1. Thread Context Switch Overhead. Here is the detail describing it. https://www.javacodegeeks.com/2015/03/jvm-pressure-context-switching-overhead.html . Pay attention to three things here, • OS may deprive thread of CPU time to make scheduling look fair to other threads. It has cost as described, especially when there are too many threads and less CPU cores. => We need powerful machine. • Blocking operation. Thread needs to yield its CPU time to other threads when blocked. It has context switch overhead too. => Currently, all RPC messages need to be put into the blocking queue first and then dispatched and consumed by multiple threads. So does the messages being sent out via outbox. • Threads contend for CPU time. If there are multiple threads being blocked on the same condition, they contend for execution when the condition is met. More threads, more contention. => less threads can make thing quick in some cases. 2. RPC message size is typically very small. And it’s processing is very quick too. After processing, thread turns back immediately waits on the same condition for new message. It’s very likely more than one thread wait on the same condition and contend each other if we configure more threads than necessary. 3. Driver’s RPC message is far more than single executor. In RPC, the chances to block and contend, the frequencies of context switch are different between driver and each executor. Thus, we need different threads configurations for driver and executors to have optimized performance.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
