[
https://issues.apache.org/jira/browse/SPARK-34098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Jie updated SPARK-34098:
-----------------------------
Description:
{code:java}
mvn clean test -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -Pmesos -Pyarn
-Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive -pl
sql/hive -Pscala-2.13
-DwildcardSuites=org.apache.spark.sql.hive.client.HadoopVersionInfoSuite
-Dtest=none
{code}
Independent test HadoopVersionInfoSuite all case passed, but execute
{code:java}
mvn clean test -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -Pmesos -Pyarn
-Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive -pl
sql/hive -Pscala-2.13
{code}
HadoopVersionInfoSuite failed as follow:
{code:java}
HadoopVersionInfoSuite: 22:32:30.310 WARN
org.apache.spark.sql.hive.client.IsolatedClientLoader: Failed to resolve Hadoop
artifacts for the version 2.7.4. We will change the hadoop version from 2.7.4
to 2.7.4 and try again. It is recommended to set jars used by Hive metastore
client through spark.sql.hive.metastore.jars in the production environment. -
SPARK-32256: Hadoop VersionInfo should be preloaded *** FAILED ***
java.lang.RuntimeException: [unresolved dependency:
org.apache.hive#hive-metastore;2.0.1: not found, unresolved dependency:
org.apache.hive#hive-exec;2.0.1: not found, unresolved dependency:
org.apache.hive#hive-common;2.0.1: not found, unresolved dependency:
org.apache.hive#hive-serde;2.0.1: not found, unresolved dependency:
com.google.guava#guava;14.0.1: not found, unresolved dependency:
org.apache.hadoop#hadoop-client;2.7.4: not found] at
org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1423)
at
org.apache.spark.sql.hive.client.IsolatedClientLoader$.$anonfun$downloadVersion$2(IsolatedClientLoader.scala:122)
at org.apache.spark.sql.catalyst.util.package$.quietly(package.scala:42) at
org.apache.spark.sql.hive.client.IsolatedClientLoader$.downloadVersion(IsolatedClientLoader.scala:122)
at
org.apache.spark.sql.hive.client.IsolatedClientLoader$.liftedTree1$1(IsolatedClientLoader.scala:75)
at
org.apache.spark.sql.hive.client.IsolatedClientLoader$.forVersion(IsolatedClientLoader.scala:63)
at
org.apache.spark.sql.hive.client.HadoopVersionInfoSuite.$anonfun$new$1(HadoopVersionInfoSuite.scala:46)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85) at
org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83) at
org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
{code}
need some investigate.
was:
{code:java}
mvn clean test -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -Pmesos -Pyarn
-Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive -pl
sql/hive -Pscala-2.13
-DwildcardSuites=org.apache.spark.sql.hive.client.HadoopVersionInfoSuite
-Dtest=none
{code}
passed, but execute
{code:java}
mvn clean test -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -Pmesos -Pyarn
-Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive -pl
sql/hive -Pscala-2.13
{code}
HadoopVersionInfoSuite failed as follow:
{code:java}
HadoopVersionInfoSuite: 22:32:30.310 WARN
org.apache.spark.sql.hive.client.IsolatedClientLoader: Failed to resolve Hadoop
artifacts for the version 2.7.4. We will change the hadoop version from 2.7.4
to 2.7.4 and try again. It is recommended to set jars used by Hive metastore
client through spark.sql.hive.metastore.jars in the production environment. -
SPARK-32256: Hadoop VersionInfo should be preloaded *** FAILED ***
java.lang.RuntimeException: [unresolved dependency:
org.apache.hive#hive-metastore;2.0.1: not found, unresolved dependency:
org.apache.hive#hive-exec;2.0.1: not found, unresolved dependency:
org.apache.hive#hive-common;2.0.1: not found, unresolved dependency:
org.apache.hive#hive-serde;2.0.1: not found, unresolved dependency:
com.google.guava#guava;14.0.1: not found, unresolved dependency:
org.apache.hadoop#hadoop-client;2.7.4: not found] at
org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1423)
at
org.apache.spark.sql.hive.client.IsolatedClientLoader$.$anonfun$downloadVersion$2(IsolatedClientLoader.scala:122)
at org.apache.spark.sql.catalyst.util.package$.quietly(package.scala:42) at
org.apache.spark.sql.hive.client.IsolatedClientLoader$.downloadVersion(IsolatedClientLoader.scala:122)
at
org.apache.spark.sql.hive.client.IsolatedClientLoader$.liftedTree1$1(IsolatedClientLoader.scala:75)
at
org.apache.spark.sql.hive.client.IsolatedClientLoader$.forVersion(IsolatedClientLoader.scala:63)
at
org.apache.spark.sql.hive.client.HadoopVersionInfoSuite.$anonfun$new$1(HadoopVersionInfoSuite.scala:46)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85) at
org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83) at
org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
{code}
need some investigate.
> HadoopVersionInfoSuite faild when maven test in Scala 2.13
> ----------------------------------------------------------
>
> Key: SPARK-34098
> URL: https://issues.apache.org/jira/browse/SPARK-34098
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 3.2.0
> Reporter: Yang Jie
> Priority: Minor
>
>
> {code:java}
> mvn clean test -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -Pmesos -Pyarn
> -Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive
> -pl sql/hive -Pscala-2.13
> -DwildcardSuites=org.apache.spark.sql.hive.client.HadoopVersionInfoSuite
> -Dtest=none
> {code}
> Independent test HadoopVersionInfoSuite all case passed, but execute
>
> {code:java}
> mvn clean test -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -Pmesos -Pyarn
> -Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive
> -pl sql/hive -Pscala-2.13
> {code}
>
> HadoopVersionInfoSuite failed as follow:
> {code:java}
> HadoopVersionInfoSuite: 22:32:30.310 WARN
> org.apache.spark.sql.hive.client.IsolatedClientLoader: Failed to resolve
> Hadoop artifacts for the version 2.7.4. We will change the hadoop version
> from 2.7.4 to 2.7.4 and try again. It is recommended to set jars used by Hive
> metastore client through spark.sql.hive.metastore.jars in the production
> environment. - SPARK-32256: Hadoop VersionInfo should be preloaded *** FAILED
> *** java.lang.RuntimeException: [unresolved dependency:
> org.apache.hive#hive-metastore;2.0.1: not found, unresolved dependency:
> org.apache.hive#hive-exec;2.0.1: not found, unresolved dependency:
> org.apache.hive#hive-common;2.0.1: not found, unresolved dependency:
> org.apache.hive#hive-serde;2.0.1: not found, unresolved dependency:
> com.google.guava#guava;14.0.1: not found, unresolved dependency:
> org.apache.hadoop#hadoop-client;2.7.4: not found] at
> org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1423)
> at
> org.apache.spark.sql.hive.client.IsolatedClientLoader$.$anonfun$downloadVersion$2(IsolatedClientLoader.scala:122)
> at org.apache.spark.sql.catalyst.util.package$.quietly(package.scala:42) at
> org.apache.spark.sql.hive.client.IsolatedClientLoader$.downloadVersion(IsolatedClientLoader.scala:122)
> at
> org.apache.spark.sql.hive.client.IsolatedClientLoader$.liftedTree1$1(IsolatedClientLoader.scala:75)
> at
> org.apache.spark.sql.hive.client.IsolatedClientLoader$.forVersion(IsolatedClientLoader.scala:63)
> at
> org.apache.spark.sql.hive.client.HadoopVersionInfoSuite.$anonfun$new$1(HadoopVersionInfoSuite.scala:46)
> at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85) at
> org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83) at
> org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
> {code}
> need some investigate.
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
