squito commented on a change in pull request #26059: [SPARK-29398][core]
Support dedicated thread pools for RPC endpoints.
URL: https://github.com/apache/spark/pull/26059#discussion_r333683292
##########
File path: core/src/main/scala/org/apache/spark/rpc/RpcEndpoint.scala
##########
@@ -146,3 +146,13 @@ private[spark] trait RpcEndpoint {
* [[ThreadSafeRpcEndpoint]] for different messages.
*/
private[spark] trait ThreadSafeRpcEndpoint extends RpcEndpoint
+
+/**
+ * An endpoint that uses a dedicated thread pool for delivering messages.
+ */
+private[spark] trait IsolatedRpcEndpoint extends RpcEndpoint {
+
+ /** How many threads to use for delivering messages. By default, use a
single thread. */
+ def threadCount(): Int = 1
Review comment:
I'm trying to wrap my head around what happens if you create an
`IsolatedRpcEndpoint` with threadCount() > 1, given the code in `Inbox` which
checks for inheritance from `ThreadSafeRpcEndpoint`:
https://github.com/apache/spark/blob/2b3c3793c97dc0e48f1e098c6f1038f51607b716/core/src/main/scala/org/apache/spark/rpc/netty/Inbox.scala#L123
I guess if you expect one endpoint to be served by multiple threads, it
makes sense you'd want `Inbox.enableConcurrent = false` and you'd have to make
your endpoint safe for that -- but worth a comment here at least.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
