Yin Huai created SPARK-11998:
--------------------------------
Summary: Branch 1.6's hadoop 2.2 tests always fail the entire
VersionsSuite
Key: SPARK-11998
URL: https://issues.apache.org/jira/browse/SPARK-11998
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.6.0
Reporter: Yin Huai
Right now, we hardcoded hadoop 2.4 to download when we need to download hive
and hadoop jars for metadata hive. However, SPARK-9596 makes isoalted client
loader share hadoop classes. So, if there is any compatibility issues between
the shared hadoop and hadoop 2.4, VersionsSuite will failed. Actually, in spark
1.6's hadoop 2.2 tests, VersionsSuite always fail
(https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Dashboard%20(Patrick)/job/Spark-1.6-SBT/AMPLAB_JENKINS_BUILD_PROFILE=hadoop2.2,label=spark-test/40/testReport/junit/org.apache.spark.sql.hive.client/).
Because {{HftpFileSystem}} has been moved to a new location in Hadoop 2.3+.
From Hadoop 2.4, we get {{org.apache.hadoop.hdfs.web.HftpFileSystem}}. But,
when we look up the class (by looking at hadoop 2.2's jars), it is not in this
location.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
