[
https://issues.apache.org/jira/browse/BIGTOP-1272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14053060#comment-14053060
]
bhashit parikh commented on BIGTOP-1272:
----------------------------------------
After much head banging against classpath errors and local hadoop config: I was
finally able to run the code using {{hadoop jar}} command. In case someone else
needs it, here are the steps that I used for running the {{mahout}} job.
# To get the appropriate mahout.jar compiled against hadoop 2.2.0 that I was
using: (cloned mahout from its git repo).
## {{mvn -Dhadoop2.version=2.2.0 -DskipTests clean package}}.
## find the {{mahout-mrlegacy-1.0-SNAPSHOT-job.jar}} jar file at
{{mrlegacy/target}} dir in your git clone dir.
# set {{HADOOP_CLASSPATH}} env variable to include the jar file built in the
previous step, plus your scala library. For ex.
{noformat}
export
HADOOP_CLASSPATH=/home/bp/jars/pig-withouthadoop.jar:/home/bp/jars/mahout-mrlegacy-1.0-SNAPSHOT-job.jar:/home/bp/opts/candidates/scala/scala-2.11.1/lib/scala-library.jar
{noformat}
Change the paths according to where you have stored the jars. I included the
pig jar as well as I wanted to run the pig cleaner as well.
# After copying the generated transactions records to HDFS, run the {{hadoop
jar}} command.
{noformat}
hadoop jar /home/bp/code/bigtop/bigtop-bigpetstore/BigPetStore.jar
org.apache.bigtop.bigpetstore.recommend.ItemRecommender
/bps_integration_/cleaned/Mahout /bps_integration_/Mahout/factorization
/bps_integration/Mahout/recommendations
{noformat}
My local hadoop installation version is 2.2.0, which is the same the version
configured in {{gradle}}.
And voila!. It works. Mahout is generating recommendations.
> BigPetStore: Productionize the Mahout recommender
> -------------------------------------------------
>
> Key: BIGTOP-1272
> URL: https://issues.apache.org/jira/browse/BIGTOP-1272
> Project: Bigtop
> Issue Type: New Feature
> Components: Blueprints
> Affects Versions: backlog
> Reporter: jay vyas
> Attachments: BIGTOP-1272.patch, BIGTOP-1272.patch, arch.jpeg
>
>
> BIGTOP-1271 adds patterns into the data that gaurantee that a meaningfull
> type of product recommendation can be given for at least *some* customers,
> since we know that there are going to be many customers who only bought 1
> product, and also customers that bought 2 or more products -- even in a
> dataset size of 10. due to the gaussian distribution of purchases that is
> also in the dataset generator.
> The current mahout recommender code is statically valid: It runs to
> completion in local unit tests if a hadoop 1x tarball is present but
> otherwise it hasn't been tested at scale. So, lets get it working. this
> JIRA also will comprise:
> - deciding wether to use mahout 2x for unit tests (default on mahout maven
> repo is the 1x impl) and wether or not bigtop should host a mahout 2x jar?
> After all, bigtop builds a mahout 2x jar as part of its packaging process,
> and BigPetStore might thus need a mahout 2x jar in order to test against the
> right same of bigtop releases.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
