Github user aarondav commented on a diff in the pull request:
https://github.com/apache/spark/pull/3734#discussion_r22092572
--- Diff:
network/shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockManager.java
---
@@ -60,8 +61,11 @@
private final TransportConf conf;
public ExternalShuffleBlockManager(TransportConf conf) {
- // TODO: Give this thread a name.
- this(conf, Executors.newSingleThreadExecutor());
+ this(conf, Executors.newSingleThreadExecutor(
--- End diff --
Actually, this should be daemon, I think, otherwise it may prevent the JVM
from exiting even when it's not doing anything. If we wanted to prevent the
service from exiting before cleanup has completed, I would rather block in a
stop() method until the queue of directories to delete has been drained, but I
do not think this too important. Unlike Spark Executors, the graceful
termination of this service should be uncorrelated with buildup of shuffle data.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
