Hi,
Thanks for the link and I looked through it, I am still having trouble
replacing the default hadoop distribution with my own optimized version
I am a bit confused over why there are so many hadoop dependencies in the
maven project, there are four artifactIds
1) hadoop-core, 2) hadoop-common, 3)hadoop-mapreduce-client-core,
4)hadoop-mapreduce-client-common
I managed to
1) install into maven local repository the hadoop-core-1.0.3.jar file that
contains modifications I made, using the following command
mvn install:install-file -DgroupId=org.apache.hadoop
-DartifactId=modified-hadoop-core
-Dversion=1.0.3 -Dpackaging=jar
-Dfile=Path/To.Jar/modified-hadoop-core-1.0.3.jar
the installation seems to have went fine,
2) Then I changed all hadoop-core dependencies in trunk/pom.xml and
trunk/core/pom.xml
I commented the original hadoop-core artifactId dependency but kept the
other three hadoop dependencies, since there is no hadoop-common-1.0.3.jar
file that exists,
This is the dependency I used to try to replace the original hadoop-core
dependency
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>modified-hadoop-core</artifactId>
<version>1.0.3</version>
</dependency>
But it still doesn't seem to work. when compiling CIMapper, it still
couldn't find my customized Mapper class that was in
modified-hadoop-core-1.0.3.jar file,
I am not sure what is the cause of this? any suggestion would be greatly
appreciated,
Thanks
Yunming
On Sun, Dec 16, 2012 at 12:21 AM, Ted Dunning <[email protected]> wrote:
> Change the pom to refer to your jar as a system dependency and insert the
> path where your jars are explicitly.
>
>
> http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#System_Dependencies
>
> On Sat, Dec 15, 2012 at 12:12 AM, Yunming Zhang
> <[email protected]>wrote:
>
> > Hi,
> >
> > I have implemented a version of Hadoop that is optimized for some common
> > machine learning algorithms, however, I am confused over how to modify
> the
> > pom.xml file to replace the default 1.0.4 Hadoop distribution for
> > compilation with my own customized hadoop distribution?
> >
> > I am able to set HADOOP_HOME variable to run Mahout on my own
> > distribution, but I want to be able to have the CIMapper in the source
> code
> > to extend my own customized mapper class, and to do that, I need to make
> > sure Mahout is compiled using my own hadoop-core.jar instead of the
> hadoop
> > 1.0.4 one that came with Mahout.
> >
> > Thanks
> >
> > Yunming
>
