Qianlan Chen created SPARK-60769:
------------------------------------
Summary: Read Maven resolution settings from session configuration
Key: SPARK-60769
URL: https://issues.apache.org/jira/browse/SPARK-60769
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 5.0.0
Reporter: Qianlan Chen
SparkSession.addArtifact is a session-level API, but its Maven resolver reads
the Ivy settings file, local repository path, and connection/read timeouts
directly from SparkContext's SparkConf. Consequently, it ignores
session-specific Maven configuration.
Read spark.jars.ivySettings, spark.jars.ivy, spark.jars.ivyConnectTimeout, and
spark.jars.ivyReadTimeout from the owning SparkSession configuration. Preserve
variable substitution, timeout defaults and validation, the optional local
repository path, and session-local runtime caches. Cluster-configured values
continue to be inherited. Existing public configuration-write rules remain in
place.
Update the configuration documentation and add regression coverage for
inherited configuration, session-specific settings and local repository
resolution, timeout parsing/defaults/validation, and session isolation.
Pull request: https://github.com/apache/spark/pull/59977
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
