[ https://issues.apache.org/jira/browse/SPARK-47010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17839707#comment-17839707 ]
Oleg Frenkel commented on SPARK-47010: -------------------------------------- Posted question on Stackoverflow: https://stackoverflow.com/questions/78366961/apache-spark-supporting-csi-driver-for-volume-type > Kubernetes: support csi driver for volume type > ---------------------------------------------- > > Key: SPARK-47010 > URL: https://issues.apache.org/jira/browse/SPARK-47010 > Project: Spark > Issue Type: New Feature > Components: Kubernetes > Affects Versions: 3.5.0 > Reporter: Oleg Frenkel > Priority: Major > > Today Spark supports the following types of Kubernetes > [volumes|https://kubernetes.io/docs/concepts/storage/volumes/]: hostPath, > emptyDir, nfs and persistentVolumeClaim. > In our case, Kubernetes cluster is multi-tenant and we cannot make > cluster-wide changes when deploying our application to the Kubernetes > cluster. Our application requires static shared file system. So, we cannot > use hostPath (don't have control of hosting VMs) and persistentVolumeClaim > (requires cluster-wide change when deploying PV). Our security department > does not allow nfs. > What would help in our case, is the use of csi driver (taken from here: > https://github.com/kubernetes-sigs/azurefile-csi-driver/blob/master/deploy/example/e2e_usage.md#option3-inline-volume): > {code:java} > kind: Pod > apiVersion: v1 > metadata: > name: nginx-azurefile-inline-volume > spec: > nodeSelector: > "kubernetes.io/os": linux > containers: > - image: mcr.microsoft.com/oss/nginx/nginx:1.19.5 > name: nginx-azurefile > command: > - "/bin/bash" > - "-c" > - set -euo pipefail; while true; do echo $(date) >> > /mnt/azurefile/outfile; sleep 1; done > volumeMounts: > - name: persistent-storage > mountPath: "/mnt/azurefile" > readOnly: false > volumes: > - name: persistent-storage > csi: > driver: file.csi.azure.com > volumeAttributes: > shareName: EXISTING_SHARE_NAME # required > secretName: azure-secret # required > mountOptions: > "dir_mode=0777,file_mode=0777,cache=strict,actimeo=30,nosharesock" # > optional {code} > > -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org