wangyum commented on code in PR #57525:
URL: https://github.com/apache/spark/pull/57525#discussion_r3658322207
##########
core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala:
##########
@@ -608,4 +609,23 @@ private[spark] object CoarseGrainedExecutorBackend extends
Logging {
// scalastyle:on println
System.exit(1)
}
+
+ /**
+ * Verify that the executor's app ID (from the --app-id launch argument)
matches the driver's
+ * app ID (from spark.app.id in the driver's SparkConf). A mismatch
indicates the executor has
+ * connected to the wrong driver, likely due to driver port reuse after the
original driver
+ * released its RPC port (e.g. via SparkContext.stop() following a fatal
error) while the
+ * driver process remained in a zombie state. Throw SparkException to
prevent the executor
+ * from registering to the wrong application and corrupting data.
+ */
+ private[spark] def verifyAppId(
+ sparkProperties: Seq[(String, String)],
+ executorAppId: String): Unit = {
+ sparkProperties.find(_._1 == "spark.app.id").map(_._2).foreach {
driverAppId =>
Review Comment:
Thank you. fix this issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
