Yang Jie created SPARK-34098:
--------------------------------
Summary: HadoopVersionInfoSuite faild when maven test in Scala 2.13
Key: SPARK-34098
URL: https://issues.apache.org/jira/browse/SPARK-34098
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 3.2.0
Reporter: Yang Jie
{code:java}
mvn clean test -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -Pmesos -Pyarn
-Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive -pl
sql/hive -Pscala-2.13
-DwildcardSuites=org.apache.spark.sql.hive.client.HadoopVersionInfoSuite
-Dtest=none
{code}
passed, but execute
{code:java}
mvn clean test -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -Pmesos -Pyarn
-Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive -pl
sql/hive -Pscala-2.13
{code}
HadoopVersionInfoSuite failed as follow:
{code:java}
HadoopVersionInfoSuite: 22:32:30.310 WARN
org.apache.spark.sql.hive.client.IsolatedClientLoader: Failed to resolve Hadoop
artifacts for the version 2.7.4. We will change the hadoop version from 2.7.4
to 2.7.4 and try again. It is recommended to set jars used by Hive metastore
client through spark.sql.hive.metastore.jars in the production environment. -
SPARK-32256: Hadoop VersionInfo should be preloaded *** FAILED ***
java.lang.RuntimeException: [unresolved dependency:
org.apache.hive#hive-metastore;2.0.1: not found, unresolved dependency:
org.apache.hive#hive-exec;2.0.1: not found, unresolved dependency:
org.apache.hive#hive-common;2.0.1: not found, unresolved dependency:
org.apache.hive#hive-serde;2.0.1: not found, unresolved dependency:
com.google.guava#guava;14.0.1: not found, unresolved dependency:
org.apache.hadoop#hadoop-client;2.7.4: not found] at
org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1423)
at
org.apache.spark.sql.hive.client.IsolatedClientLoader$.$anonfun$downloadVersion$2(IsolatedClientLoader.scala:122)
at org.apache.spark.sql.catalyst.util.package$.quietly(package.scala:42) at
org.apache.spark.sql.hive.client.IsolatedClientLoader$.downloadVersion(IsolatedClientLoader.scala:122)
at
org.apache.spark.sql.hive.client.IsolatedClientLoader$.liftedTree1$1(IsolatedClientLoader.scala:75)
at
org.apache.spark.sql.hive.client.IsolatedClientLoader$.forVersion(IsolatedClientLoader.scala:63)
at
org.apache.spark.sql.hive.client.HadoopVersionInfoSuite.$anonfun$new$1(HadoopVersionInfoSuite.scala:46)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85) at
org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83) at
org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
{code}
need some investigate.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
