nonsleepr commented on a change in pull request #26000:
[SPARK-29330][CORE][YARN] Allow users to chose the name of Spark Shuffle service
URL: https://github.com/apache/spark/pull/26000#discussion_r331619497
##########
File path:
common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java
##########
@@ -136,7 +136,11 @@
private DB db;
public YarnShuffleService() {
- super("spark_shuffle");
+ this("spark_shuffle");
Review comment:
It *is* hardcoded *here*. HDP hardcodes another value though
(`spark2_shuffle`). While vanilla Spark would keep working as is and would use
the name `spark_shuffle`, the new configuration option would allow users to
point Spark to non-vanilla shuffle service.
The changes to that class are done only to test that changing the name of
the service and in the configuration play nicely together.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
