[
https://issues.apache.org/jira/browse/SPARK-44345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-44345.
-----------------------------------
Fix Version/s: 4.0.0
Resolution: Fixed
Issue resolved by pull request 41906
[https://github.com/apache/spark/pull/41906]
> Only log unknown shuffle map output as error when shuffle migration disabled
> ----------------------------------------------------------------------------
>
> Key: SPARK-44345
> URL: https://issues.apache.org/jira/browse/SPARK-44345
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 3.4.1
> Reporter: Zhongwei Zhu
> Assignee: Zhongwei Zhu
> Priority: Minor
> Labels: pull-request-available
> Fix For: 4.0.0
>
>
> When decommission and shuffle migration is enabled, there're lots of error
> message like `Asked to update map output for unknown shuffle` .
> As shuffle clean and unregister is done by ContextCleaner in an async way,
> when target block manager received the shuffle block from decommissioned
> block manager, then update shuffle location to map output tracker, but at
> that time, shuffle might have been unregistered. This should not consider as
> error.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
