Hi I have managed to pull down the latest trunk from (https://github.com/apache/mahout) but I now having another problem with the build from this latest branch ("trunk"). Please see below for my steps and errors.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ $ sudo git clone https://github.com/apache/mahout mahout-latest Cloning into 'mahout-latest'... remote: Reusing existing pack: 76730, done. remote: Total 76730 (delta 0), reused 0 (delta 0) Receiving objects: 100% (76730/76730), 49.16 MiB | 193 KiB/s, done. Resolving deltas: 100% (34682/34682), done. warning: remote HEAD refers to nonexistent ref, unable to checkout. $ cd mahout-latest/ $ sudo git branch -a remotes/origin/mahout-0.1 remotes/origin/mahout-0.2 remotes/origin/mahout-0.8 remotes/origin/trunk $ sudo rm -rf mahout-latest $ sudo git clone -b trunk https://github.com/apache/mahout mahout-latest Cloning into 'mahout-latest'... remote: Reusing existing pack: 76730, done. remote: Total 76730 (delta 0), reused 0 (delta 0) Receiving objects: 100% (76730/76730), 49.16 MiB | 194 KiB/s, done. Resolving deltas: 100% (34682/34682), done. Checking out files: 100% (1475/1475), done. mvn -X install Apache Maven 3.2.1 (ea8b2b07643dbb1b84b6d16e1f08391b666bc1e9; 2014-02-15T04:37:52+10:00) Maven home: /usr/local/apache-maven/apache-maven-3.2.1 Java version: 1.6.0_32, vendor: Sun Microsystems Inc. Java home: /usr/java/jdk1.6.0_32/jre Default locale: en_US, platform encoding: UTF-8 OS name: "linux", version: "2.6.18-274.el5", arch: "amd64", family: "unix" [INFO] Error stacktraces are turned on. [DEBUG] Reading global settings from /usr/local/apache-maven/apache-maven-3.2.1/conf/settings.xml [DEBUG] Reading user settings from /home/oracle/.m2/settings.xml [DEBUG] Using local repository at /home/oracle/.m2/repository [DEBUG] Using manager EnhancedLocalRepositoryManager with priority 10.0 for /home/oracle/.m2/repository [INFO] Scanning for projects... [DEBUG] Extension realms for project org.apache.mahout:mahout:pom:1.0-SNAPSHOT: (none) [DEBUG] Looking up lifecyle mappings for packaging pom from ClassRealm[plexus.core, parent: null] [DEBUG] Extension realms for project org.apache.mahout:mahout-buildtools:jar:1.0-SNAPSHOT: (none) [DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[plexus.core, parent: null] [DEBUG] Extension realms for project org.apache.mahout:mahout-math:jar:1.0-SNAPSHOT: (none) [DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[plexus.core, parent: null] [DEBUG] Extension realms for project org.apache.mahout:mahout-core:jar:1.0-SNAPSHOT: (none) [DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[plexus.core, parent: null] [DEBUG] Extension realms for project org.apache.mahout:mahout-integration:jar:1.0-SNAPSHOT: (none) [DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[plexus.core, parent: null] [DEBUG] Extension realms for project org.apache.mahout:mahout-examples:jar:1.0-SNAPSHOT: (none) [DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[plexus.core, parent: null] [DEBUG] Extension realms for project org.apache.mahout:mahout-distribution:pom:1.0-SNAPSHOT: (none) [DEBUG] Looking up lifecyle mappings for packaging pom from ClassRealm[plexus.core, parent: null] [DEBUG] Extension realms for project org.apache.mahout:mahout-math-scala:jar:1.0-SNAPSHOT: (none) [DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[plexus.core, parent: null] [DEBUG] Extension realms for project org.apache.mahout:mahout-spark:jar:1.0-SNAPSHOT: (none) [DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[plexus.core, parent: null] [WARNING] [WARNING] Some problems were encountered while building the effective model for org.apache.mahout:mahout-spark:jar:1.0-SNAPSHOT [WARNING] 'build.plugins.plugin.version' for org.scala-tools:maven-scala-plugin is missing. @ line 108, column 15 [WARNING] The expression ${pom.version} is deprecated. Please use ${project.version} instead. [WARNING] The expression ${pom.version} is deprecated. Please use ${project.version} instead. [WARNING] The expression ${pom.version} is deprecated. Please use ${project.version} instead. [WARNING] The expression ${pom.version} is deprecated. Please use ${project.version} instead. [WARNING] [WARNING] It is highly recommended to fix these problems because they threaten the stability of your build. [WARNING] [WARNING] For this reason, future Maven versions might no longer support building such malformed projects. [WARNING] ................ CUT OFF LINES .... [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Mahout Build Tools ................................ FAILURE [ 0.991 s] [INFO] Apache Mahout ..................................... SKIPPED [INFO] Mahout Math ....................................... SKIPPED [INFO] Mahout Core ....................................... SKIPPED [INFO] Mahout Integration ................................ SKIPPED [INFO] Mahout Examples ................................... SKIPPED [INFO] Mahout Release Package ............................ SKIPPED [INFO] Mahout Math/Scala wrappers ........................ SKIPPED [INFO] Mahout Spark bindings ............................. SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2.495 s [INFO] Finished at: 2014-04-04T14:04:56+10:00 [INFO] Final Memory: 9M/245M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-remote-resources-plugin:1.1:process (default) on project mahout-buildtools: Error finding remote resources manifests: /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-latest/buildtools/target/maven-shared-archive-resources/META-INF/NOTICE (No such file or directory) -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-remote-resources-plugin:1.1:process (default) on project mahout-buildtools: Error finding remote resources manifests at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:108) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:76) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:116) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:361) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:155) at org.apache.maven.cli.MavenCli.execute(MavenCli.java:584) at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:213) at org.apache.maven.cli.MavenCli.main(MavenCli.java:157) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) Caused by: org.apache.maven.plugin.MojoExecutionException: Error finding remote resources manifests at org.apache.maven.plugin.resources.remote.ProcessRemoteResourcesMojo.processResourceBundles(ProcessRemoteResourcesMojo.java:1185) at org.apache.maven.plugin.resources.remote.ProcessRemoteResourcesMojo.execute(ProcessRemoteResourcesMojo.java:500) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:133) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208) ... 19 more Caused by: java.io.FileNotFoundException: /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-latest/buildtools/target/maven-shared-archive-resources/META-INF/NOTICE (No such file or directory) at java.io.FileOutputStream.open(Native Method) at java.io.FileOutputStream.<init>(FileOutputStream.java:194) at java.io.FileOutputStream.<init>(FileOutputStream.java:145) at java.io.FileWriter.<init>(FileWriter.java:73) at org.apache.maven.plugin.resources.remote.ProcessRemoteResourcesMojo.processResourceBundles(ProcessRemoteResourcesMojo.java:1093) ... 22 more [ERROR] [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Thanks and Regards, Truong Phan P + 61 2 8576 5771 M + 61 4 1463 7424 E [email protected] W www.telstra.com -----Original Message----- From: Phan, Truong Q [mailto:[email protected]] Sent: Friday, 4 April 2014 11:03 AM To: [email protected] Subject: RE: Mahout v0.9 is not working with 2.2.0-cdh5.0.0-beta-1 Hi Gokhan, I having difficult to checkout the latest trunk using MVN. Here is my checkout issue. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ $ sudo git config --global http.proxy http://58.162.7.28:3128 $ sudo git config --global http.sslVerify false $ sudo git clone https://github.com/apache/mahout Cloning into 'mahout'... remote: Reusing existing pack: 76730, done. remote: Total 76730 (delta 0), reused 0 (delta 0) Receiving objects: 100% (76730/76730), 49.16 MiB | 191 KiB/s, done. Resolving deltas: 100% (34682/34682), done. warning: remote HEAD refers to nonexistent ref, unable to checkout. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Thanks and Regards, Truong Phan P + 61 2 8576 5771 M + 61 4 1463 7424 E [email protected] W www.telstra.com -----Original Message----- From: Gokhan Capan [mailto:[email protected]] Sent: Friday, 4 April 2014 5:50 AM To: [email protected] Subject: Re: Mahout v0.9 is not working with 2.2.0-cdh5.0.0-beta-1 Could you checkout the latest trunk and build the project again? Sent from my iPhone > On Apr 2, 2014, at 9:50, Sean Owen <[email protected]> wrote: > > Not sure what's going on there. Your build is somehow internally > inconsistent at runtime since it fails on the client side due to > mismatching Hadoop versions. I can start making up increasingly > unlikely causes, like you are building against local copies of the > artifacts that are old? Running clean and install should help if so. > > You might debug the script to print exactly what it is running. > On Apr 2, 2014 7:59 AM, "Phan, Truong Q" > <[email protected]> > wrote: > >> The build was successful with those "warning". >> >> I could not run a test case from this website: >> http://girlincomputerscience.blogspot.com.au/2010/11/apache-mahout.ht >> ml >> >> Here are my run with the error: >> >> >> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> $ pwd >> >> /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distri >> bution-0.9 >> >> $ echo $CLASSPATH >> >> $ echo $PATH >> >> /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distri >> bution-0.9/bin:/usr/local/apache-maven/apache-maven-3.2.1/bin:/usr/lo >> cal/bin:/bin:/usr/bin:/home/oracle/bin:.:/usr/java/jdk1.6.0_32/bin >> >> $ echo $JAVA_HOME >> /usr/java/jdk1.6.0_32 >> >> $ env | grep HAD >> HADOOP_CMD=/usr/bin/hadoop >> HADOOP_HOME=/usr/lib/hadoop >> >> HADOOP_STREAMING=/usr/lib/hadoop-mapreduce/hadoop-streaming-2.2.0-cdh >> 5.0.0-beta-1.jar >> >> $ mahout recommenditembased --input mydata.dat --usersFile user.dat >> --numRecommendations 2 --output output/ --similarityClassname >> SIMILARITY_PEARSON_CORRELATION Running on hadoop, using >> /usr/lib/hadoop/bin/hadoop and HADOOP_CONF_DIR= >> MAHOUT-JOB: >> /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distri >> bution-0.9/examples/target/mahout-examples-0.9-job.jar >> 14/04/02 16:52:48 INFO common.AbstractJob: Command line arguments: >> {--booleanData=[false], --endPhase=[2147483647], >> --input=[mydata.dat], --maxPrefsInItemSimilarity=[500], >> --maxPrefsPerUser=[10], --maxSimilaritiesPerItem=[100], >> --minPrefsPerUser=[1], --numRecommendations=[2], --output=[output/], >> --similarityClassname=[SIMILARITY_PEARSON_CORRELATION], >> --startPhase=[0], --tempDir=[temp], --usersFile=[user.dat]} >> 14/04/02 16:52:48 INFO common.AbstractJob: Command line arguments: >> {--booleanData=[false], --endPhase=[2147483647], >> --input=[mydata.dat], --minPrefsPerUser=[1], >> --output=[temp/preparePreferenceMatrix], >> --ratingShift=[0.0], --startPhase=[0], --tempDir=[temp]} >> 14/04/02 16:52:49 INFO Configuration.deprecation: mapred.input.dir is >> deprecated. Instead, use mapreduce.input.fileinputformat.inputdir >> 14/04/02 16:52:49 INFO Configuration.deprecation: >> mapred.compress.map.output is deprecated. Instead, use >> mapreduce.map.output.compress >> 14/04/02 16:52:49 INFO Configuration.deprecation: mapred.output.dir >> is deprecated. Instead, use >> mapreduce.output.fileoutputformat.outputdir >> Exception in thread "main" java.lang.IncompatibleClassChangeError: >> Found interface org.apache.hadoop.mapreduce.JobContext, but class was >> expected >> at >> org.apache.mahout.common.HadoopUtil.getCustomJobName(HadoopUtil.java:174) >> at >> org.apache.mahout.common.AbstractJob.prepareJob(AbstractJob.java:614) >> at >> org.apache.mahout.cf.taste.hadoop.preparation.PreparePreferenceMatrixJob.run(PreparePreferenceMatrixJob.java:73) >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >> at >> org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.run(RecommenderJob.java:164) >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >> at >> org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.main(RecommenderJob.java:322) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at >> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72) >> at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144) >> at >> org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:152) >> at >> org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at org.apache.hadoop.util.RunJar.main(RunJar.java:212) >> oracle@bpdevdmsdbs01: ---- >> /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distri >> bution-0.9 >> ----- >> >> >> Thanks and Regards, >> Truong Phan >> >> >> P + 61 2 8576 5771 >> M + 61 4 1463 7424 >> E [email protected] >> W www.telstra.com >> >> >> >> -----Original Message----- >> From: Sean Owen [mailto:[email protected]] >> Sent: Wednesday, 2 April 2014 4:05 PM >> To: Mahout User List >> Subject: Re: Mahout v0.9 is not working with 2.2.0-cdh5.0.0-beta-1 >> >> Hm, OK something sounds wrong with your directory structure, given >> the warnings. I assumed this was changed. It could be that the >> .tar.gz distribution isn't quite correctly set up for building from source. >> >> The compilation here is nothing to do with Hadoop. You show a >> successful build; what's the part that fails? >> >> On Wed, Apr 2, 2014 at 6:59 AM, Phan, Truong Q < >> [email protected]> wrote: >>> Where did I modifying the build? >>> Here are my steps of the build. >>> I got the source from one of the official mirror website and build it. >>> The only one exception here is that I am using the Cloudera CDH 5.0. >>> This latest CDHv5.0 might not work with Mahout v0.9. >>> >>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> Install Mahout >>> >>> $ javac -version >>> javac 1.6.0_32 >>> >>> $ cd /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout >>> >>> $ wget >>> http://mirror.mel.bkb.net.au/pub/apache/mahout/0.9/mahout-distributi >>> on >>> -0.9-src.tar.gz >>> >>> $ mv mahout-distribution-0.9 mahout-distribution-0.9.old >>> >>> $ tar xvf mahout-distribution-0.9-src.tar.gz >>> >>> $ cd mahout-distribution-0.9 >>> >>> $ ls >>> bin buildtools core distribution examples integration >>> LICENSE.txt math math-scala NOTICE.txt pom.xml README.txt src >>> >>> $ mvn clean install -Dhadoop2.version=2.2.0-cdh5.0.0-beta-1 >> -DskipTests=true [INFO] Scanning for projects... >>> ............ >>> >>> [INFO] >>> -------------------------------------------------------------------- >>> -- >>> -- >>> [INFO] Reactor Summary: >>> [INFO] >>> [INFO] Mahout Build Tools ................................ SUCCESS [ >>> 7.235 s] [INFO] Apache Mahout ..................................... >>> SUCCESS [ 1.017 s] [INFO] Mahout Math >>> ....................................... SUCCESS [15:46 min] [INFO] >>> Mahout Core ....................................... SUCCESS [24:29 >>> min] [INFO] Mahout Integration ................................ >>> SUCCESS [03:38 min] [INFO] Mahout Examples >>> ................................... SUCCESS [02:40 min] [INFO] >>> Mahout Release Package ............................ SUCCESS [ 0.075 >>> s] [INFO] Mahout Math/Scala wrappers ........................ >>> SUCCESS >>> [01:12 min] [INFO] >>> -------------------------------------------------------------------- >>> -- >>> -- >>> [INFO] BUILD SUCCESS >>> [INFO] >>> -------------------------------------------------------------------- >>> -- >>> -- >>> [INFO] Total time: 47:57 min >>> [INFO] Finished at: 2014-04-02T15:07:50+10:00 [INFO] Final Memory: >>> 49M/288M [INFO] >>> -------------------------------------------------------------------- >>> -- >>> -- >>> >>> >>> Thanks and Regards, >>> Truong Phan >>> >>> >>> P + 61 2 8576 5771 >>> M + 61 4 1463 7424 >>> E [email protected] >>> W www.telstra.com >>> >>> >>> >>> -----Original Message----- >>> From: Sean Owen [mailto:[email protected]] >>> Sent: Wednesday, 2 April 2014 3:33 PM >>> To: Mahout User List >>> Subject: Re: Mahout v0.9 is not working with 2.2.0-cdh5.0.0-beta-1 >>> >>> This may be getting to you're-on-your-own-territory since you're >> modifying the build. This error means your directory structure >> doesn't match up with declarations. You said somewhere that the >> parent of module X was Y, but the location given points to the pom of a >> module that isn't Y. >>> >>> On Wed, Apr 2, 2014 at 5:28 AM, Phan, Truong Q < >> [email protected]> wrote: >>>> Hi Sean, >>>> >>>> I am trying to build the Mahout again and got some WARNINGs so far. >>>> Can you give me some hints what I have done wrong here? >>>> >>>> Thanks for your help so far. >>
