[
https://issues.apache.org/jira/browse/SPARK-35416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18058206#comment-18058206
]
Dongjoon Hyun commented on SPARK-35416:
---------------------------------------
It seems that you are suffering due to a well-known K8s {{ReadWriteOnce}} bug.
Apache Spark 4.0+ uses `ReadWriteOncePod` by default.
*
[https://spark.apache.org/docs/latest/core-migration-guide.html#upgrading-from-core-35-to-40]
{quote} * Since Spark 4.0, Spark uses {{ReadWriteOncePod}} instead of
{{ReadWriteOnce}} access mode in persistence volume claims. To restore the
legacy behavior, you can set
{{spark.kubernetes.legacy.useReadWriteOnceAccessMode}} to {{{}true{}}}.
{quote}
> Support PersistentVolumeClaim Reuse
> -----------------------------------
>
> Key: SPARK-35416
> URL: https://issues.apache.org/jira/browse/SPARK-35416
> Project: Spark
> Issue Type: Improvement
> Components: Kubernetes
> Affects Versions: 3.2.0
> Reporter: Dongjoon Hyun
> Assignee: Dongjoon Hyun
> Priority: Major
> Fix For: 3.2.0
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
