tgravescs commented on a change in pull request #26000:
[SPARK-29330][CORE][YARN] Allow users to chose the name of Spark Shuffle service
URL: https://github.com/apache/spark/pull/26000#discussion_r333691945
##########
File path:
common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java
##########
@@ -136,7 +136,11 @@
private DB db;
public YarnShuffleService() {
- super("spark_shuffle");
+ this("spark_shuffle");
+ }
+
+ protected YarnShuffleService(String serviceName) {
Review comment:
So the name by itself isn't going to be enough. If you really want it
configurable we are going to have to have the port configurable. For instance
the config name for the port spark.shuffle.service.port needs to be able to be
something like spark.shuffle.service.{serviceName}.port. Otherwise all the
spark shuffle servers will try to get the same port and fail. The only other
option will be to use 0 for ephemeral but
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
