weixiuli commented on code in PR #36141:
URL: https://github.com/apache/spark/pull/36141#discussion_r851714246
##########
core/src/main/scala/org/apache/spark/shuffle/ShuffleBlockPusher.scala:
##########
@@ -129,11 +130,8 @@ private[spark] class ShuffleBlockPusher(conf: SparkConf)
extends Logging {
try {
pushUpToMax()
} catch {
- case e: FileNotFoundException =>
- logWarning("The shuffle files got deleted when this
shuffle-block-push-thread " +
- "was reading from them which could happen when the job finishes and
the driver " +
- "instructs the executor to cleanup the shuffle. In this case, push
of the blocks " +
- "belonging to this shuffle will stop.", e)
+ case NonFatal(e) =>
+ logWarning("Failed to push blocks to a remote shuffle service", e)
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
