This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.
from a97885b [SPARK-33350][SHUFFLE] Add support to DiskBlockManager to
create merge directory and to get the local shuffle merged data
add cf07036 [SPARK-35593][K8S][CORE] Support shuffle data recovery on the
reused PVCs
No new revisions were added by this update.
Summary of changes:
.../scala/org/apache/spark/MapOutputTracker.scala | 22 ++-
.../shuffle/KubernetesLocalDiskShuffleDataIO.scala | 24 +--
...ernetesLocalDiskShuffleExecutorComponents.scala | 102 ++++++++++
.../KubernetesLocalDiskShuffleDataIOSuite.scala | 219 +++++++++++++++++++++
4 files changed, 353 insertions(+), 14 deletions(-)
copy core/src/main/scala/org/apache/spark/shuffle/ShuffleWriter.scala =>
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/shuffle/KubernetesLocalDiskShuffleDataIO.scala
(53%)
create mode 100644
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/shuffle/KubernetesLocalDiskShuffleExecutorComponents.scala
create mode 100644
resource-managers/kubernetes/core/src/test/scala/org/apache/spark/shuffle/KubernetesLocalDiskShuffleDataIOSuite.scala
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
