mridulm commented on code in PR #41906:
URL: https://github.com/apache/spark/pull/41906#discussion_r1278819867
##########
core/src/main/scala/org/apache/spark/MapOutputTracker.scala:
##########
@@ -805,8 +808,10 @@ private[spark] class MapOutputTrackerMaster(
shuffleStatuses.get(shuffleId) match {
case Some(shuffleStatus) =>
shuffleStatus.updateMapOutput(mapId, bmAddress)
+ case None if shuffleMigrationEnabled =>
+ logDebug(s"Asked to update map output for unknown shuffle
${shuffleId}")
case None =>
- logError(s"Asked to update map output for unknown shuffle
${shuffleId}")
+ logWarn(s"Asked to update map output for unknown shuffle ${shuffleId}")
Review Comment:
Let us not change this.
The discussion above was whether we should make something which was `error`
earlier into `debug` or not.
Note - just because migration is enabled does not tell us if this failure is
due to migration - it need not be.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
