akirillov commented on a change in pull request #25500: [SPARK-28778][MESOS]
Fixed executors advertised address when running in virtual network
URL: https://github.com/apache/spark/pull/25500#discussion_r316467338
##########
File path:
resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala
##########
@@ -268,27 +268,19 @@ private[spark] class MesosCoarseGrainedSchedulerBackend(
.getOrElse {
throw new SparkException(s"Executor Spark home `$EXECUTOR_HOME` is
not set!")
}
- val runScript = new File(executorSparkHome, "./bin/spark-class").getPath
- command.setValue(
- "%s \"%s\" org.apache.spark.executor.CoarseGrainedExecutorBackend"
- .format(prefixEnv, runScript) +
- s" --driver-url $driverURL" +
- s" --executor-id $taskId" +
- s" --hostname ${executorHostname(offer)}" +
- s" --cores $numCores" +
- s" --app-id $appId")
+ val executable = new File(executorSparkHome, "./bin/spark-class").getPath
+ val runScript = "%s \"%s\"
org.apache.spark.executor.CoarseGrainedExecutorBackend"
Review comment:
There was a problem with interpolation - checkstyle was failing on the
following string:
```
val runScript = s"$prefixEnv \"$executable\"
org.apache.spark.executor.CoarseGrainedExecutorBackend"
```
Error:
```
error
file=/Users/akirillov/projects/mesosphere/spark/resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala
message=Expected token SEMI but got Token(STRING_LITERAL,"
org.apache.spark.executor.CoarseGrainedExecutorBackend",10094,"
org.apache.spark.executor.CoarseGrainedExecutorBackend")
```
The working variant with interpolation looks as following:
```
val runScript =
s"""$prefixEnv "$executable"
| org.apache.spark.executor.CoarseGrainedExecutorBackend
|""".stripMargin.replaceAll("\n", " ")
```
not sure if it's better than `String.format`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
