Hello Curtis Try the following:
hadoop jar ./target/distributed-shell.jar.rebuilt-one org.apache.hadoop.yarn.applications.distributedshell.Client -jar ... If you are running hadoop without the jar command, it will find the first instance of Client.class in its classpath which I am guessing is from the jars used to start the cluster and not the re-built jar. -- Hitesh On Jun 21, 2013, at 4:09 PM, Curtis Ullerich wrote: > I should clarify that the build doesn't actually fail (so that bug doesn't > seem to be the issue). It reports success. But then the output from the > just-build jar doesn't change. > > Here's my output from building: > > C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn > clean package -DskipTests > [INFO] Scanning for projects... > [INFO] > [INFO] > ------------------------------------------------------------------------ > [INFO] Building hadoop-yarn-applications-distributedshell 3.0.0-SNAPSHOT > [INFO] > ------------------------------------------------------------------------ > [INFO] > [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ > hadoop-yarn-applications-distributedshell --- > [INFO] Deleting > C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target > [INFO] > [INFO] --- maven-antrun-plugin:1.6:run (create-testdirs) @ > hadoop-yarn-applications-distributedshell --- > [INFO] Executing tasks > > main: > [mkdir] Created dir: > C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-dir > [INFO] Executed tasks > [INFO] > [INFO] --- maven-resources-plugin:2.2:resources (default-resources) @ > hadoop-yarn-applications-distributedshell --- > [INFO] Using default encoding to copy filtered resources. > [INFO] > [INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ > hadoop-yarn-applications-distributedshell --- > [INFO] Compiling 3 source files to > C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\classes > [INFO] > [INFO] --- maven-resources-plugin:2.2:testResources (default-testResources) @ > hadoop-yarn-applications-distributedshell --- > [INFO] Using default encoding to copy filtered resources. > [INFO] > [INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ > hadoop-yarn-applications-distributedshell --- > [INFO] Compiling 1 source file to > C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-classes > [INFO] > [INFO] --- maven-jar-plugin:2.3.1:jar (default) @ > hadoop-yarn-applications-distributedshell --- > [INFO] Building jar: > C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar > [INFO] > [INFO] --- maven-surefire-plugin:2.12.3:test (default-test) @ > hadoop-yarn-applications-distributedshell --- > [INFO] Tests are skipped. > [INFO] > [INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ > hadoop-yarn-applications-distributedshell --- > [INFO] > [INFO] --- maven-site-plugin:3.0:attach-descriptor (attach-descriptor) @ > hadoop-yarn-applications-distributedshell --- > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 3.487s > [INFO] Finished at: Fri Jun 21 16:08:02 PDT 2013 > [INFO] Final Memory: 19M/367M > [INFO] > ------------------------------------------------------------------------ > > > > On Fri, Jun 21, 2013 at 3:57 PM, Arpit Agarwal <[email protected]> > wrote: > Curtis, if you are running 'mvn clean' in a subproject you may be hitting > this bug: > https://issues.apache.org/jira/browse/HADOOP-9383 > > You might have better luck if you include install along with package in the > same invocation. > > -Arpit > > > On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich <[email protected]> > wrote: > I've executed the commands as you've said, and the jar that is run is > unchanged. Here's exactly what I did. > > C:\l\> start-dfs > C:\l\> start-yarn > C:\l\> cd > hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell > C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell> > hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar > target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar > -shell_command whoami > ...output... > ...modify logging code in > C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java > C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell> > mvn clean install -DskipTests > C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell> > hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar > target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar > -shell_command whoami > ...identical output to before... > > Do you see anything wrong with what I've done? > > Thanks, > Curtis > > > > > On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <[email protected]> wrote: > Hi Curtis, > > where are you picking your jar file from? once you run above command you will > see the updated jar file in > "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar" > > I hope you are not using below jar file > "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar" > > > What chris has said is right. > > once you have taken latest code, you should follow below steps > 1) mvn clean install -DskipTests (clean to remove previously generated code) > 2) now say you are updating distributed shell client code. then go to > ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/" > and then run "mvn clean install" and use jar from target sub folder. > > Thanks, > Omkar Joshi > Hortonworks Inc. > > > On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <[email protected]> > wrote: > Hi Chris, > > I really appreciate the response. What you described is what I initially > tried. The changes never seem to take effect though. Here's what I've done > (this is Windows): > > cd %hadoop_install_dir% > mvn clean package -DskipTests > mvn install -DskipTests > > --- modify the code in distributed shell's Client.java --- > > cd > hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell > mvn clean > mvn package -DskipTests > mvn install -DskipTests > > Then I run the jar, just as before. I've just been changing log statements to > see if the changes worked. They don't--the output doesn't change after doing > this rebuild. I've also tried clearing the files put on HDFS in my user > directory in case that was the issue. I've taken a more thorough look at > BUILDING.txt and I seemed to be consistent with the procedures described > there. Am I missing anything else? I've tried restarting yarn and dfs, though > I didn't think that would matter. > > Thanks, > Curtis > > > > > On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <[email protected]> > wrote: > Hi Curtis, > > I handle this by running "mvn install -DskipTests" once from the root of the > whole hadoop project to install the sub-modules in my local Maven repository. > Then, you can change your working directory to any sub-module (like > distributed shell) and run a successful Maven build for just that sub-module. > Full details are in the BUILDING.txt file in the root of the hadoop project. > Look for the section titled "Building components separately". I hope this > helps. > > Chris Nauroth > Hortonworks > http://hortonworks.com/ > > > > On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <[email protected]> > wrote: > What is the way to compile just a subproject of Hadoop with Maven (without > rebuilding parent modules)? In my case I'm wondering about DistributedShell. > I thought that this would have worked but no changes seem to take effect > after building this way. > > Thanks, > Curtis > > > > > >
