Hu Liu, created BIGTOP-1470:
-------------------------------
Summary: Mahout 0.9 couldn't work hadoop 2.x
Key: BIGTOP-1470
URL: https://issues.apache.org/jira/browse/BIGTOP-1470
Project: Bigtop
Issue Type: Bug
Components: build
Affects Versions: 0.8.0
Reporter: Hu Liu,
Priority: Blocker
I built Mahout 0.9 with Bigtop and found it couldn't work with hadoop 2.x.
{code}
Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
interface org.apache.hadoop.mapreduce.JobContext, but class was expected
at org.apache.mahout.common.HadoopUtil.getCustomJobName(HadoopUtil.java:174)
at org.apache.mahout.common.AbstractJob.prepareJob(AbstractJob.java:614)
at
org.apache.mahout.cf.taste.hadoop.preparation.PreparePreferenceMatrixJob.run(PreparePreferenceMatrixJob.java:73)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.run(RecommenderJob.java:164)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.main(RecommenderJob.java:322)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:152)
at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
{code}
The root cause is that bigtop uses the wrong parameter hadoop2.version when
building Mahout.
{code}
mvn clean install -Dmahout.skip.distribution=false -DskipTests
-Dhadoop2.version=$HADOOP_VERSION "$@"
{code}
For Mahout 0.9 , the correct property is hadoop.version in pom.xml of Mahout 0.9
{code}
<hadoop.version>1.2.1</hadoop.version>
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
