Hi Chris, Appreciate your help, and sorry for the crossposting on both 'user' and 'yarn-dev'. I first posted on yarn-dev and didn't see anything. I then thought I might not be able to post on that list. That's why I posted it again on 'user'. Should I post this kind of questions on 'yarn-dev' or 'user'?
Right now, I cannot compile hadoop-yarn-server-resourcemanager independently. I have to use mvn package -Pdist -DskipTests -Dtar to compile the whole project. What command is appropriate to this scenario? Thanks, oxhead On Sat, Apr 13, 2013 at 12:41 AM, Chris Nauroth <[email protected]>wrote: > I don't have an answer to your exact question, but I do have a different > suggestion that prevents the need to do frequent rebuilds of the whole > Hadoop source tree. First, do a full build of the distribution tar.gz. > Extract it and set up a custom hadoop-env.sh for yourself. Inside the > hadoop-env.sh file, export the environment variables > HADOOP_USER_CLASSPATH_FIRST=true and HADOOP_CLASSPATH= any classpath that > you want to prepend before the classes loaded from the distribution. For > example, this is what I have in mine right now, because I'm mostly working > on HDFS and NodeManager: > > export HADOOP_USER_CLASSPATH_FIRST=true > HADOOP_REPO=~/git/hadoop-common > export > HADOOP_CLASSPATH=$HADOOP_REPO/hadoop-common-project/hadoop-common/target/classes:$HADOOP_REPO/hadoop-hdfs-project/hadoop-hdfs/target/classes:$HADOOP_REPO/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes:$HADOOP_REPO/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/classes > > For your ResourceManager work, you could set up your HADOOP_CLASSPATH to > point at your hadoop-yarn-server-resourcemanager/target/classes directory. > Then, source (.) this hadoop-env.sh in any shell that you're using to run > hadoop commands. The daemons will print their full classpath before > launching, so you can check that to see if it worked. > > With all of this in place, you can keep recompiling just > hadoop-yarn-server-resourcemanager whenever you make changes instead of the > whole hadoop-common tree. Does this help? > > Thanks, > --Chris > > > On Fri, Apr 12, 2013 at 8:46 PM, Chin-Jung Hsu <[email protected]>wrote: > >> I am implementing my own YARN scheduler under 2.0.3-alpha. Is that >> possible to build only the ResourceManager project, and then create a >> distribution tar.gz for the entire Hadoop project? Right now, the >> compiling time takes me about 9 minutes. >> >> Thanks, >> oxhead >> > >
