[
https://issues.apache.org/jira/browse/SPARK-59966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-59966:
-----------------------------------
Labels: pull-request-available (was: )
> Do not append the default remote repository when custom Ivy settings are used
> for Hive metastore jars
> -----------------------------------------------------------------------------------------------------
>
> Key: SPARK-59966
> URL: https://issues.apache.org/jira/browse/SPARK-59966
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.4.0
> Reporter: Qianlan Chen
> Priority: Major
> Labels: pull-request-available
>
> After SPARK-56867, `IsolatedClientLoader` respects `spark.jars.ivySettings`
> when resolving Hive metastore jars.
> However, it reads `spark.sql.maven.additionalRemoteRepositories` using
> `SparkConf.get(ConfigEntry)`. This applies the default Google-hosted Maven
> Central mirror even when the user did not explicitly configure additional
> repositories.
> `MavenUtils.loadIvySettings` then appends that repository to the resolver
> chain loaded from `ivysettings.xml`.
> As a result, `ivysettings.xml` does not fully control which repositories are
> contacted. This can cause unexpected network access in environments that
> intend to use only private or authenticated repositories.
> Expected behavior:
> * If `spark.jars.ivySettings` is set and
> `spark.sql.maven.additionalRemoteRepositories` is not explicitly set, use
> only the resolvers from `ivysettings.xml`.
> * If both configurations are explicitly set, append the configured
> additional repositories.
> * If `spark.jars.ivySettings` is not set, preserve the existing default
> repository behavior for compatibility.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
