sarutak opened a new pull request, #55881: URL: https://github.com/apache/spark/pull/55881
### What changes were proposed in this pull request? This PR modifies `IsolatedClientLoader.downloadVersion` to respect the `spark.jars.ivySettings` configuration when resolving Hive metastore jars via `spark.sql.hive.metastore.jars=maven`. Previously, `downloadVersion` always called `MavenUtils.buildIvySettings` directly, ignoring any custom Ivy settings file. With this change, if `spark.jars.ivySettings` is set, `MavenUtils.loadIvySettings` is used instead, consistent with how `DependencyUtils.resolveMavenDependencies` handles `--packages`. ### Why are the changes needed? When `spark.sql.hive.metastore.jars=maven` is configured, Spark downloads Hive metastore jars using Apache Ivy. However, unlike `--packages` / `spark.jars.packages`, this code path did not honor `spark.jars.ivySettings`. This made it impossible to download Hive metastore jars from authenticated private repositories, since credentials can only be configured through an Ivy settings file. ### Does this PR introduce _any_ user-facing change? Yes. Users who set `spark.jars.ivySettings` can now have that configuration apply to Hive metastore jar resolution as well. Previously, the Ivy settings file was only used for `spark.jars.packages`. ### How was this patch tested? Confirmed it works with a private repository which requires authentication. ### Was this patch authored or co-authored using generative AI tooling? Kiro CLI / Opus 4.7 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
