warrenzhu25 opened a new pull request #30282: URL: https://github.com/apache/spark/pull/30282
### What changes were proposed in this pull request? Add config spark.yarn.pyspark.archives to provide location of pyspark.zip and py4j.zip. ### Why are the changes needed? 1. If I have different version of `spark.yarn.archive` from current machine, then there is no way to provide matched pyspark version. This happens in version upgrade. 2. This can save effort of uploading pyspark.zip and py4j.zip every time. ### Does this PR introduce _any_ user-facing change? Yes. Added in yarn doc. ### How was this patch tested? Manually. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
