I don't have an answer to your exact question, but I do have a different suggestion that prevents the need to do frequent rebuilds of the whole Hadoop source tree. First, do a full build of the distribution tar.gz. Extract it and set up a custom hadoop-env.sh for yourself. Inside the hadoop-env.sh file, export the environment variables HADOOP_USER_CLASSPATH_FIRST=true and HADOOP_CLASSPATH= any classpath that you want to prepend before the classes loaded from the distribution. For example, this is what I have in mine right now, because I'm mostly working on HDFS and NodeManager:
export HADOOP_USER_CLASSPATH_FIRST=true HADOOP_REPO=~/git/hadoop-common export HADOOP_CLASSPATH=$HADOOP_REPO/hadoop-common-project/hadoop-common/target/classes:$HADOOP_REPO/hadoop-hdfs-project/hadoop-hdfs/target/classes:$HADOOP_REPO/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes:$HADOOP_REPO/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/classes For your ResourceManager work, you could set up your HADOOP_CLASSPATH to point at your hadoop-yarn-server-resourcemanager/target/classes directory. Then, source (.) this hadoop-env.sh in any shell that you're using to run hadoop commands. The daemons will print their full classpath before launching, so you can check that to see if it worked. With all of this in place, you can keep recompiling just hadoop-yarn-server-resourcemanager whenever you make changes instead of the whole hadoop-common tree. Does this help? Thanks, --Chris On Fri, Apr 12, 2013 at 8:46 PM, Chin-Jung Hsu <[email protected]>wrote: > I am implementing my own YARN scheduler under 2.0.3-alpha. Is that > possible to build only the ResourceManager project, and then create a > distribution tar.gz for the entire Hadoop project? Right now, the > compiling time takes me about 9 minutes. > > Thanks, > oxhead >
