rvesse commented on a change in pull request #23546: [SPARK-23153][K8s] Support client dependencies with a HCFS URL: https://github.com/apache/spark/pull/23546#discussion_r248625547
########## File path: docs/running-on-kubernetes.md ########## @@ -400,7 +415,6 @@ There are several Spark on Kubernetes features that are currently being worked o Some of these include: * Dynamic Resource Allocation and External Shuffle Service -* Local File Dependency Management Review comment: We have environments where there is nothing remotely HDFS like available and the systems are typically air-gapped so using external services like S3 isn't an option either. Primary storage is usually a high performance parallel file system (Lustre or IBM Spectrum Scale) which is just a POSIX compliant file system mounted to all nodes over the system interconnect. Using `hostPath` volume mounts isn't a realistic option either because these environments have strict security requirements. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
