ocworld opened a new pull request #32397: URL: https://github.com/apache/spark/pull/32397
### What changes were proposed in this pull request? Supporting '--packages' in the k8s cluster mode ### Why are the changes needed? In spark 3, '--packages' in the k8s cluster mode is not supported. I expected that managing dependencies by using packages like spark 2. ### Does this PR introduce _any_ user-facing change? Unlike spark 2, resolved jars are added not in cluster mode spark submit but in driver. It's because in spark 3, the feature is added that is uploading jars with prefix "file://" to s3. So, if resolved jars are added in spark submit, every jars from packages are uploading to s3! When I tested it, it is very bad experience to me. ### How was this patch tested? In my k8s environment, i tested the code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
