jimmajure opened a new issue #9181: URL: https://github.com/apache/airflow/issues/9181
**Description** In the KubernetesExecutor configuration, allow the DAGs, logs, plugins folders to be supported by a single PersistentVolumeClaim. Currently the configuration forces a separate PVC for each directory. **Use case / motivation** Mounting volumes via a PVC is an expensive task that requires multiple calls to the K8S API server, so it is beneficial to be able to mount fewer PVC volumes for each pod. We have run into scalability issues in AWS EKS when using EFS drives to support DAGs and logs. The configuration should allow users to mount as many PVC mounts as required. If people choose to put DAGs, logs and plugins in a single PVC mount, that should be supported. If they choose to put each in a separate PVC mount, that can be supported also. This should be able to be supported by separating the purpose of the mount from the mount, itself. Simply allow one or more mounts and let the user map the directories to the appropriate folder, irrespective of which mount it is on. **Related Issues** ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
