dongjoon-hyun commented on a change in pull request #25849: [SPARK-29175][SQL]
Make maven central repository in IsolatedClientLoader configurable
URL: https://github.com/apache/spark/pull/25849#discussion_r326468590
##########
File path:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala
##########
@@ -112,18 +115,18 @@ private[hive] object IsolatedClientLoader extends
Logging {
private def downloadVersion(
version: HiveVersion,
hadoopVersion: String,
- ivyPath: Option[String]): Seq[URL] = {
+ ivyPath: Option[String],
+ centralRepo: String): Seq[URL] = {
val hiveArtifacts = version.extraDeps ++
Seq("hive-metastore", "hive-exec", "hive-common", "hive-serde")
.map(a => s"org.apache.hive:$a:${version.fullVersion}") ++
Seq("com.google.guava:guava:14.0.1",
s"org.apache.hadoop:hadoop-client:$hadoopVersion")
-
val classpath = quietly {
SparkSubmitUtils.resolveMavenCoordinates(
hiveArtifacts.mkString(","),
SparkSubmitUtils.buildIvySettings(
- Some("http://www.datanucleus.org/downloads/maven2";),
Review comment:
@xuanyuanking . If then, could you make a separate JIRA and PR for this line
change with the following description?
> The repository currently used is
"http://www.datanucleus.org/downloads/maven2";, which is no longer maintained.
This will sometimes cause downloading failure and make hive test cases flaky.
End users can also set this config to the central repository they want to
access.
Then, we can backport your new PR to branch-2.4, too. After that, we can
proceed this PR on top of that. That will be very helpful for our LTS branch
`branch-2.4`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
