[
https://issues.apache.org/jira/browse/SPARK-47010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17839707#comment-17839707
]
Oleg Frenkel edited comment on SPARK-47010 at 11/3/25 3:00 PM:
---------------------------------------------------------------
Posted question on Stackoverflow:
[https://stackoverflow.com/questions/78366961/apache-spark-supporting-csi-driver-for-volume-type]
As of Nov 3, 2025, the post on Stackoverflow is hidden due to no activity.
was (Author: ofrenkel):
Posted question on Stackoverflow:
https://stackoverflow.com/questions/78366961/apache-spark-supporting-csi-driver-for-volume-type
> Kubernetes: support csi driver for volume type
> ----------------------------------------------
>
> Key: SPARK-47010
> URL: https://issues.apache.org/jira/browse/SPARK-47010
> Project: Spark
> Issue Type: New Feature
> Components: Kubernetes
> Affects Versions: 3.5.0
> Reporter: Oleg Frenkel
> Priority: Major
>
> Today Spark supports the following types of Kubernetes
> [volumes|https://kubernetes.io/docs/concepts/storage/volumes/]: hostPath,
> emptyDir, nfs and persistentVolumeClaim.
> In our case, Kubernetes cluster is multi-tenant and we cannot make
> cluster-wide changes when deploying our application to the Kubernetes
> cluster. Our application requires static shared file system. So, we cannot
> use hostPath (don't have control of hosting VMs) and persistentVolumeClaim
> (requires cluster-wide change when deploying PV). Our security department
> does not allow nfs.
> What would help in our case, is the use of csi driver (taken from here:
> https://github.com/kubernetes-sigs/azurefile-csi-driver/blob/master/deploy/example/e2e_usage.md#option3-inline-volume):
> {code:java}
> kind: Pod
> apiVersion: v1
> metadata:
> name: nginx-azurefile-inline-volume
> spec:
> nodeSelector:
> "kubernetes.io/os": linux
> containers:
> - image: mcr.microsoft.com/oss/nginx/nginx:1.19.5
> name: nginx-azurefile
> command:
> - "/bin/bash"
> - "-c"
> - set -euo pipefail; while true; do echo $(date) >>
> /mnt/azurefile/outfile; sleep 1; done
> volumeMounts:
> - name: persistent-storage
> mountPath: "/mnt/azurefile"
> readOnly: false
> volumes:
> - name: persistent-storage
> csi:
> driver: file.csi.azure.com
> volumeAttributes:
> shareName: EXISTING_SHARE_NAME # required
> secretName: azure-secret # required
> mountOptions:
> "dir_mode=0777,file_mode=0777,cache=strict,actimeo=30,nosharesock" #
> optional {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
