srowen commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-492243806 I see, the point is that multiple jobs might want to share some files. That's the flipside to the problem that isolation solves -- you don't worry about old files lying around. It's a use case though. Do we do that elsewhere in Spark? I really dont' know. Like if you add .py files to a job and they get shipped off to YARN, are they reusable or referenceable? Consistency might be the best argument one way or the other. Well, I end up neutral on it. I'd probably suggest budging towards making the subdir to reach consensus here, absent any other information or input.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
