[
https://issues.apache.org/jira/browse/SPARK-46091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-46091:
-----------------------------------
Labels: pull-request-available (was: )
> [KUBERNETES] Respect the existing kubernetes container SPARK_LOCAL_DIRS env
> ---------------------------------------------------------------------------
>
> Key: SPARK-46091
> URL: https://issues.apache.org/jira/browse/SPARK-46091
> Project: Spark
> Issue Type: Improvement
> Components: Kubernetes
> Affects Versions: 3.5.0
> Reporter: Fei Wang
> Priority: Major
> Labels: pull-request-available
>
> Respect the user defined SPARK_LOCAL_DIRS container env when setup local dirs.
>
> For example, we use hostPath for spark local dir.
> But we do not mount the sub disks directly to the pod, we mount a root path
> for spark driver/executor pod.
>
> For example, the root path is `/hadoop`.
>
> And there are sub disks under that, likes `hadoop/1, /hadoop/2, /hadoop/3,
> /hadoop4`.
>
> And we want to define the SPARK_LOCAL_DIRS in the driver/executor pod env.
>
> But now, the user specified SPARK_LOCAL_DIRS does not work.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
