This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new 166df8b1df59 [SPARK-46344][CORE] Warn properly when a driver exists successfully but master is disconnected 166df8b1df59 is described below commit 166df8b1df5965fe3f15fecd5574545746b0b18f Author: Dongjoon Hyun <dh...@apple.com> AuthorDate: Sat Dec 9 15:33:48 2023 -0800 [SPARK-46344][CORE] Warn properly when a driver exists successfully but master is disconnected ### What changes were proposed in this pull request? This PR aims to warn properly when a driver exists successfully but master is disconnected. ### Why are the changes needed? In this case, `Master` considers them `Error` eventually. ![Screenshot 2023-12-09 at 3 05 27 PM](https://github.com/apache/spark/assets/9700541/1323819b-4a0c-466d-afaa-845f507a905e) **Worker Log** ``` 23/12/09 15:13:21 INFO Worker: Driver driver-20231209151301-0003 exited successfully === Master is disconnected here === 23/12/09 15:13:53 WARN Worker: Driver driver-20231209151332-0004 exited successfully while master is disconnected. === A new master starts and is connected here === 23/12/09 15:17:10 INFO Worker: Driver driver-20231209151707-0000 exited successfully ``` ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #44278 from dongjoon-hyun/SPARK-46344. Authored-by: Dongjoon Hyun <dh...@apple.com> Signed-off-by: Dongjoon Hyun <dh...@apple.com> --- core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala b/core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala index 1422a1484f8d..785129e1d818 100755 --- a/core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala +++ b/core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala @@ -878,7 +878,12 @@ private[deploy] class Worker( case DriverState.FAILED => logWarning(s"Driver $driverId exited with failure") case DriverState.FINISHED => - logInfo(s"Driver $driverId exited successfully") + registrationRetryTimer match { + case Some(_) => + logWarning(s"Driver $driverId exited successfully while master is disconnected.") + case _ => + logInfo(s"Driver $driverId exited successfully") + } case DriverState.KILLED => logInfo(s"Driver $driverId was killed by user") case _ => --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org