koodin9 opened a new pull request, #36884: URL: https://github.com/apache/spark/pull/36884
### What changes were proposed in this pull request? Adds hive resource URL of origLoader, the existing classloader, to the newly created classloader of IsolatedClientLoader. ### Why are the changes needed? When "spark.sql.hive.metastore.jars" is set to "path", IsolatedClientLoader can refer to jars corresponding to a specific hive metastore version with viewfs or hdfs schema. At this time, there is no problem when the deploy-mode is client, but in the case of cluster, the reference to the xml files related to the hive setting disappears, and then an error occurs in the process of initializing HiveConf in HiveClientImpl. The xml configuration files related to hive are set to "spark.yarn.dist.files". At this time, these files, such as hive-site.xml, are saved as local files in the local usercache of the YARN Nodemanager host. The problem is solved by storing the URL pointing to the location of these resources and adding it to the list of resources loaded by the newly created IsolatedClientLoader. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? * test by unit test * I tested a sample spark application on a cluster with the build version of this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
