Yeah, this was one of my thoughts with MAHOUT-185 - turn some of our Driver classes to just fire off a Tool. It is very convenient to be able to do this, and it's becoming more standard as well.
I need to dig up my stuff in decomposer/contrib-hadoop and pull that in and integrate it with Drew's patch on that ticket. -jake On Tue, Jan 26, 2010 at 7:19 PM, Robin Anil <robin.a...@gmail.com> wrote: > Mahout algorithms are not using ToolRunner of Hadoop. I guess many core > hadoop-ers like that feature. I think we should be supporting that feature > by 0.3 > > > Robin > > On Wed, Jan 27, 2010 at 5:59 AM, Sean Owen <sro...@gmail.com> wrote: > > > These look like Hadoop params, to the hadoop command? why wouldn't > > hadoop be parsing those, or, why would the Job command have to shuttle > > them to Hadoop? I thought these were typically set in the config .xml > > files anyhow. > > > > On Tue, Jan 26, 2010 at 11:43 PM, Aurora Skarra-Gallagher > > <aur...@yahoo-inc.com> wrote: > > > Hi, > > > > > > I'm using the PFPGrowth code ( > > http://issues.apache.org/jira/browse/MAHOUT-157) from Mahout 0.3 and it > > works fine on my local box. However, when I try to get it to run on our > grid > > cluster, it amazingly does not allow any parameters to be passed to > Hadoop. > > When I look at the code > > > (mahout/core/src/main/java/org/apache/mahout/fpm/pfpgrowth/PFPGrowth.java), > > I see that there is no way to pass custom configuration parameters (like > - > > Dmapred.job.queue.name=X or -libjars or any other parameter for that > > matter). > > > > > > I am shocked that it would be done this way. To get this to work, I > need > > to go change the actual PFPGrowth.java file, add my conf.set("key", > "val") > > lines, and recompile. Is there any other way to do this? Why would it be > > written in such a way that all hadoop parameters are disallowed? > > > > > > Thanks, > > > Aurora > > > > > > > > > -- > ------ > Robin Anil > Blog: http://techdigger.wordpress.com > ------- > Try out Swipeball for iPhone > Video: http://www.youtube.com/watch?v=3hvEbWHciwU > iTunes: http://itunes.com/apps/swipeball >
