Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.
The following page has been changed by NigelDaley: http://wiki.apache.org/hadoop/HudsonBuildServer The comment on the change is: Hudson server info is now maintained elsewhere. Attempting to add a redirect. ------------------------------------------------------------------------------ - = Introducing Hudson = + #REFRESH 0 /general/Hudson - We use [https://hudson.dev.java.net/ Hudson] as our continuous integration server: - - * [http://hudson.zones.apache.org/hudson/] - - Hudson runs automated patch verification builds for Hadoop. - - = Administering Hudson = - - Hudson is run by the {{{hudson}}} user on {{{hudson.zones.apache.org}}}. Any Apache committer may have an account on this machine. To manage the Hudson server, log into this machine and - - {{{sudo su - hudson}}} - - This will give you the proper shell environment to manage the Hudson server. Hudson uses a number of tools for itself and for the builds it runs. All tools are located in {{{~hudson/tools}}}. Hudson stores all of it's config files and builds in {{{$HUDSON_HOME}}} which is set to {{{~hudson/hudson}}}. - - '''Note:''' Please notify others on the Hadoop IRC channel (#hadoop at irc.freenode.net) before you start manipulating Hudson, to make sure that you're not stepping on someone else's toes. Any substantial configuration changes to Hadoop builds should be discussed on the developer mailing list. - - == Tasks == - - Most tasks, such as creating a new build or modifying an existing build, are performed through the Hudson web interface. Once you login to Hudson, you are provided with an expanded number of administration options. To create a login on Hudson, see below. The following tasks must be performed after logging into {{{hudson.zones.apache.org}}} and switching to the {{{hudson}}} user. - - 1. '''Stopping a nightly build:''' - When logged into Hudson, a red X button is displayed next to each running build. Clicking the button will end the build as far as Hudson is concerned, but the underlying processes sometimes don't get properly terminated. A better approach is to kill the {{{hudsonBuildHadoopNightly.sh}}} process or, if that doesn't exist, the {{{ant}}} process: - 1. Login to {{{hudson.zones.apache.org}}} - 1. Switch to the {{{hudson}}} user - 1. {{{ps -ef | grep hudson}}} to get the {{{hudsonBuildHadoopNightly.sh}}} or {{{ant}}} process id - 1. {{{kill [-9] <hudsonBuildHadoopNightly.sh or ant process id>}}} - 1. {{{ps -ef | grep hudson}}} again to verify the build and test processes are gone. - When this process dies, Hudson will fail the build. - - 1. '''Restarting a hung patch testing queue:''' - A number of issues can cause the [http://hudson.zones.apache.org/hudson/view/Hadoop/job/Hadoop-Patch-Admin/lastSuccessfulBuild/artifact/HADOOP_PatchQueue.html queue of pending patches] to hang or appear hung. Note that the queue page is updated by the [http://hudson.zones.apache.org/hudson/view/Hadoop/job/Hadoop-Patch-Admin/ Hadoop-Patch-Admin] build every five minutes. Sometimes, this build can't run for a long period of time because other build are running on Hudson. - 1. '''There are patches in the queue but [http://hudson.zones.apache.org/hudson/view/Hadoop/job/Hadoop-Patch/ Hadoop-Patch] build is not running''' - Try kicking off the [http://hudson.zones.apache.org/hudson/view/Hadoop/job/Hadoop-Patch-Admin/ Hadoop-Patch-Admin] manually. If that doesn't start the [http://hudson.zones.apache.org/hudson/view/Hadoop/job/Hadoop-Patch/ Hadoop-Patch] build, then kick off the [http://hudson.zones.apache.org/hudson/view/Hadoop/job/Hadoop-Patch/ Hadoop-Patch] build manually. That should be all that is necessary in this scenario. - 1. '''The [http://hudson.zones.apache.org/hudson/view/Hadoop/job/Hadoop-Patch/ Hadoop-Patch] is running well over 2 hours''' - Some test has likely hung. See "Killing a hung test" below. - - 1. '''Killing a hung test:''' - Sometimes it is desirable to just kill a hung test and allow the rest of the build to continue. Although the build will still fail, it can sometimes provide useful information if you can get a thread dump on the hung test and, if the build is in the first test pass, the second pass might succeed as some tests hang only intermittently. You can tell a test is hung by looking at the console log for the build and if the last entry is some time ago (at least 15 minutes) it is safe to assume the test is hung. - 1. {{{ssh hudson.zones.apache.org}}} - 1. {{{sudo su - hudson}}} - 1. Find the hung test: {{{ps -laxww | grep hudson | grep test.build.data | grep -v grep'}}} - For example: - - {{{ - 0 1000 17700 4921 0 53 20150292 1032 ffffffffbbf4f456 S ? 0:00 /export/home/hudson/tools/java/jdk1.5.0_11/jre/bin/java -Xmx256m -Dtest.build.data=/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/build/test/data -Dtest.cache.data=/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/build/test/cache -Dhadoop.log.dir=/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/build/test/logs -Dtest.src.dir=/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/src/test -Djava.library.path=/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/build/native/ - SunOS-x86-32/lib:/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/lib/native/ - SunOS-x86-32 -Dinstall.c++.examples=/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/ - trunk/build/c++-examples/SunOS-x86-32 -classpath /export/home/hudson/hudson/jobs/Hadoop-Patch/ - workspace/trunk/build/test/classes:/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/ - src/test:/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/build:/export/home/hudson/ - hudson/jobs/Hadoop-Patch/workspace/trunk/build/examples:/export/home/hudson/hudson/jobs/ - Hadoop-Patch/workspace/trunk/build/classes:/export/home/hudson/hudson/jobs/Hadoop-Patch/ - workspace/trunk/lib/commons-cli-2.0-SNAPSHOT.jar:/export/home/hudson/hudson/jobs/Hadoop-Patch/ - workspace/trunk/lib/commons-codec-1.3.jar:/export/home/hudson/hudson/jobs/Hadoop-Patch/ - workspace/trunk/lib/commons-httpclient-3.0.1.jar:/export/home/hudson/hudson/jobs/Hadoop-Patch/ - workspace/trunk/lib/commons-logging-1.0.4.jar:/export/home/hudson/hudson/jobs/Hadoop-Patch/ - workspace/trunk/lib/commons-logging-api-1.0.4.jar:/export/home/hudson/hudson/jobs/Hadoop-Patch/ - workspace/trunk/lib/jets3t-0.5.0.jar:/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/ - lib/jetty-5.1.4.jar:/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/lib/jetty-ext/ - commons-el.jar:/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/lib/jetty-ext/ - jasper-compiler.jar:/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/lib/jetty-ext/ - jasper-runtime.jar:/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/lib/jetty-ext/ - jsp-api.jar:/export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/lib/junit-3.8.1.jar: - /export/home/hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/lib/log4j-1.2.13.jar:/export/home/ - hudson/hudson/jobs/Hadoop-Patch/workspace/trunk/lib/servlet-api.jar:/export/home/hudson/hudson/ - jobs/Hadoop-Patch/workspace/trunk/lib/xmlenc-0.52.jar:/export/home/hudson/hudson/jobs/ - Hadoop-Patch/workspace/trunk/conf:/export/home/hudson/tools/ant/latest/lib/junit-3.8.1.jar:/export/ - home/hudson/tools/ant/apache-ant-1.6.5/lib/ant-launcher.jar:/export/home/hudson/tools/ant/latest/lib/ - ant.jar:/export/home/hudson/tools/ant/latest/lib/ant-junit.jar - org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner - org.apache.hadoop.mapred.TestMiniMRWithDFS filtertrace=true haltOnError=false haltOnFailure=false formatter=org.apache.tools.ant.taskdefs.optional.junit.SummaryJUnitResultFormatter showoutput=true formatter=org.apache.tools.ant.taskdefs.optional.junit.XMLJUnitResultFormatter,/export/home/hudson/ - hudson/jobs/Hadoop-Patch/workspace/trunk/build/test/ - TEST-org.apache.hadoop.mapred.TestMiniMRWithDFS.xml propsfile=/export/home/hudson/hudson/ - jobs/Hadoop-Patch/workspace/trunk/junit1559750177.properties - }}} - - 1. {{{kill -QUIT 17700}}} to try to get a thread dump. Although this does not always work (because the test is just too locked up), if it does work, it can provide useful debugging information. - 1. {{{kill -9 17700}}} to kill the hung test. If you are watching the console (and you should be), you should see the console output resume. - - 1. '''Creating a login on hudson.zones.apache.org and the Hudson web UI:''' - Committers may have a login on the hudson.zones.apache.org build machine. Follow [http://wiki.apache.org/general/Hudson these] directions. -
