10110346 commented on issue #23975: [SPARK-27056][MESOS]Remove start-shuffle-service.sh URL: https://github.com/apache/spark/pull/23975#issuecomment-469935278 > > > Isn't this used by standalone? I admit I haven't kept up with how the shuffle service is or isn't launched by various RMs (YARN doesn't use it right?) but are you sure it was Mesos-only? Thanks @srowen I got some information from these two PR: 1. #4990, `start-shuffle-service.sh` was added in this PR, this PR description is for Mesos to add this file. 2. #7881, `start-mesos-shuffle-service.sh` was added in this PR, this PR solves some problems for Mesos Also, I found this description in the `job-scheduling.md (L80)`: ` In standalone mode, simply start your workers with spark.shuffle.service.enabled set to true.` In fact, in standalone mode, if we started workers and we didn't set spark.shuffle.service.enabled to true ,and then we can run `$SPARK_HOME/sbin/start-shuffle-service.sh` to start shuffle service. In this case, we only need to modify the description of `job-scheduling.md (L80)`.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
