wuyi created SPARK-48637:
----------------------------
Summary: On-demand shuffle migration peer refresh during
decommission
Key: SPARK-48637
URL: https://issues.apache.org/jira/browse/SPARK-48637
Project: Spark
Issue Type: Sub-task
Components: Spark Core
Affects Versions: 3.4.3, 3.3.4, 3.5.1, 3.2.4, 3.1.3, 4.0.0
Reporter: wuyi
Currently the shuffle migration peers is refreshed every 30s by default. It
could be more effecient if we refresh the peer immediately once there is a peer
aborted.
(Strictly speaking, we only wait 30s to refresh the new peer when the there no
queued peers (i.e., `ShuffleMigrationRunnable`) in the `shuffleMigrationPool`.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
