Would make logical sense Markus, thank you. I think it's about time to try a more generic Jenkins build configuration e.g. build on Ubuntu slaves as well as Solaris. I'll see what we can get running over the next while.
On Thu, Nov 3, 2011 at 11:43 AM, Markus Jelsma <markus.jel...@openindex.io>wrote: > DiskCheckerException usually smells like running out of disk space in the > designated tmp dir. > > On Thursday 03 November 2011 12:39:11 Lewis John Mcgibbney wrote: > > Hi, > > > > If you look at the recently failing Nutch trunk builds, namely #1645, > #1650 > > & #1651, a common denominator is the > > org.apache.nutch.segment.TestSegmentMerger.testLargeMerge< > https://builds.ap > > > ache.org/job/Nutch-trunk/1651/testReport/org.apache.nutch.segment/TestSegme > > ntMerger/testLargeMerge/> which failes with the following: > > > > 2011-11-03 04:17:37,478 WARN mapred.LocalJobRunner > > (LocalJobRunner.java:run(256)) - job_local_0001 > > org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find > > > taskTracker/jobcache/job_local_0001/attempt_local_0001_m_000000_0/output/fi > > le.out in any of the configured local directories > > at > > > org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToR > > ead(LocalDirAllocator.java:389) at > > > org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocato > > r.java:138) at > > > org.apache.hadoop.mapred.MapOutputFile.getOutputFile(MapOutputFile.java:50 > > ) at > > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:193) > > > > > > Any clues as to this pattern of failures? Is it possible to nail it down > to > > the Solaris slave build environment? > > > > On Thu, Nov 3, 2011 at 4:18 AM, Apache Jenkins Server < > > > > jenk...@builds.apache.org> wrote: > > > See <https://builds.apache.org/job/Nutch-trunk/1651/changes> > > > > > > Changes: > > > > > > [lewismc] commit to make trivial change to working directory > > > configuration for ant sonar target... > > > > > > [lewismc] commit to add MySQL driver classpath to sonar analysis target > > > > > > [ferdy] NUTCH-1192 Add '/runtime' to svn ignore > > > > > > ------------------------------------------ > > > [...truncated 2838 lines...] > > > > > > compile: > > > [echo] Compiling plugin: urlfilter-validator > > > > > > [javac] > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/src/plugin/build-plugin.xml:117: warning: 'includeantruntime' > was > > > not set, defaulting to > > > build.sysclasspath=last; set to false for repeatable builds > > > > > > [javac] Compiling 1 source file to > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/urlfilter-validator/classes > > > > > > jar: > > > [jar] Building jar: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/urlfilter-validator/urlfilter-validator.jar > > > > > > deps-test: > > > > > > deploy: > > > [copy] Copying 1 file to > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/plugins/urlfilter-validator > > > > > > copy-generated-lib: > > > [copy] Copying 1 file to > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/plugins/urlfilter-validator > > > > > > init: > > > [mkdir] Created dir: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/urlmeta > > > > > > [mkdir] Created dir: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/urlmeta/classes > > > > > > [mkdir] Created dir: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/urlmeta/test > > > > > > [mkdir] Created dir: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/plugins/urlmeta > > > > > > init-plugin: > > > > > > deps-jar: > > > > > > clean-lib: > > > > > > resolve-default: > > > [ivy:resolve] :: loading settings :: file = > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/ivy/ivysettings.xml > > > > > > compile: > > > [echo] Compiling plugin: urlmeta > > > > > > [javac] > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/src/plugin/build-plugin.xml:117: warning: 'includeantruntime' > was > > > not set, defaulting to > > > build.sysclasspath=last; set to false for repeatable builds > > > > > > [javac] Compiling 2 source files to > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/urlmeta/classes > > > > > > jar: > > > [jar] Building jar: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/urlmeta/urlmeta.jar > > > > > > deps-test: > > > > > > deploy: > > > [copy] Copying 1 file to > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/plugins/urlmeta > > > > > > copy-generated-lib: > > > [copy] Copying 1 file to > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/plugins/urlmeta > > > > > > init: > > > [mkdir] Created dir: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/urlnormalizer-basic > > > > > > [mkdir] Created dir: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/urlnormalizer-basic/classes > > > > > > [mkdir] Created dir: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/urlnormalizer-basic/test > > > > > > [mkdir] Created dir: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/plugins/urlnormalizer-basic > > > > > > init-plugin: > > > > > > deps-jar: > > > > > > clean-lib: > > > > > > resolve-default: > > > [ivy:resolve] :: loading settings :: file = > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/ivy/ivysettings.xml > > > > > > compile: > > > [echo] Compiling plugin: urlnormalizer-basic > > > > > > [javac] > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/src/plugin/build-plugin.xml:117: warning: 'includeantruntime' > was > > > not set, defaulting to > > > build.sysclasspath=last; set to false for repeatable builds > > > > > > [javac] Compiling 1 source file to > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/urlnormalizer-basic/classes > > > > > > jar: > > > [jar] Building jar: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/urlnormalizer-basic/urlnormalizer-basic.jar > > > > > > deps-test: > > > > > > deploy: > > > [copy] Copying 1 file to > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/plugins/urlnormalizer-basic > > > > > > copy-generated-lib: > > > [copy] Copying 1 file to > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/plugins/urlnormalizer-basic > > > > > > init: > > > [mkdir] Created dir: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/urlnormalizer-pass > > > > > > [mkdir] Created dir: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/urlnormalizer-pass/classes > > > > > > [mkdir] Created dir: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/urlnormalizer-pass/test > > > > > > [mkdir] Created dir: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/plugins/urlnormalizer-pass > > > > > > init-plugin: > > > > > > deps-jar: > > > > > > clean-lib: > > > > > > resolve-default: > > > [ivy:resolve] :: loading settings :: file = > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/ivy/ivysettings.xml > > > > > > compile: > > > [echo] Compiling plugin: urlnormalizer-pass > > > > > > [javac] > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/src/plugin/build-plugin.xml:117: warning: 'includeantruntime' > was > > > not set, defaulting to > > > build.sysclasspath=last; set to false for repeatable builds > > > > > > [javac] Compiling 1 source file to > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/urlnormalizer-pass/classes > > > > > > jar: > > > [jar] Building jar: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/urlnormalizer-pass/urlnormalizer-pass.jar > > > > > > deps-test: > > > > > > deploy: > > > [copy] Copying 1 file to > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/plugins/urlnormalizer-pass > > > > > > copy-generated-lib: > > > [copy] Copying 1 file to > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/plugins/urlnormalizer-pass > > > > > > [mkdir] Created dir: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/urlnormalizer-regex/test/data > > > > > > [copy] Copying 4 files to > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/urlnormalizer-regex/test/data > > > > > > init: > > > [mkdir] Created dir: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/urlnormalizer-regex/classes > > > > > > [mkdir] Created dir: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/plugins/urlnormalizer-regex > > > > > > init-plugin: > > > > > > deps-jar: > > > > > > clean-lib: > > > > > > resolve-default: > > > [ivy:resolve] :: loading settings :: file = > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/ivy/ivysettings.xml > > > > > > compile: > > > [echo] Compiling plugin: urlnormalizer-regex > > > > > > [javac] > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/src/plugin/build-plugin.xml:117: warning: 'includeantruntime' > was > > > not set, defaulting to > > > build.sysclasspath=last; set to false for repeatable builds > > > > > > [javac] Compiling 1 source file to > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/urlnormalizer-regex/classes > > > > > > [javac] Note: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > > k/trunk/src/plugin/urlnormalizer-regex/src/java/org/apache/nutch/net/urln > > > ormalizer/regex/RegexURLNormalizer.java uses unchecked or unsafe > > > operations. > > > > > > [javac] Note: Recompile with -Xlint:unchecked for details. > > > > > > jar: > > > [jar] Building jar: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/urlnormalizer-regex/urlnormalizer-regex.jar > > > > > > deps-test: > > > > > > init: > > > > > > init-plugin: > > > > > > clean-lib: > > > > > > resolve-default: > > > [ivy:resolve] :: loading settings :: file = > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/ivy/ivysettings.xml > > > > > > compile: > > > > > > jar: > > > > > > deps-test: > > > > > > deploy: > > > > > > copy-generated-lib: > > > > > > deploy: > > > [copy] Copying 1 file to > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/plugins/urlnormalizer-regex > > > > > > copy-generated-lib: > > > [copy] Copying 1 file to > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/plugins/urlnormalizer-regex > > > > > > compile: > > > > > > job: > > > [jar] Building jar: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/nutch-2011-11-03_04-11-51.job > > > > > > resolve-test: > > > [taskdef] Could not load definitions from resource > > > > > > org/sonar/ant/antlib.xml. It could not be found. > > > > > > copy-libs: > > > > > > compile-core-test: > > > [javac] > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build.xml:320: warning: 'includeantruntime' was not set, > > > defaulting to > > > build.sysclasspath=last; set to false for repeatable builds > > > > > > [javac] Compiling 32 source files to > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/test/classes > > > > > > [javac] Note: Some input files use or override a deprecated API. > > > [javac] Note: Recompile with -Xlint:deprecation for details. > > > [javac] Note: Some input files use unchecked or unsafe operations. > > > [javac] Note: Recompile with -Xlint:unchecked for details. > > > > > > test-core: > > > [mkdir] Created dir: > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/test/data > > > > > > [copy] Copying 90 files to > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/test/data > > > > > > [copy] Copying 1 file to > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/test/classes > > > > > > [copy] Copying 1 file to > > > > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build/test/classes > > > > > > [junit] WARNING: multiple versions of ant detected in path for junit > > > [junit] > > > > > > > jar:file:/export/home/hudson/tools/ant/latest/lib/ant.jar!/org/apache/to > > > ols/ant/Project.class > > > > > > [junit] and > > > > > > > jar:file:/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/N > > > > utch-trunk/trunk/build/lib/ant-1.6.5.jar!/org/apache/tools/ant/Project.cl > > > ass > > > > > > [junit] Running org.apache.nutch.crawl.TestCrawlDbMerger > > > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 7.829 > sec > > > [junit] Running org.apache.nutch.crawl.TestGenerator > > > [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 60.748 > sec > > > [junit] Running org.apache.nutch.crawl.TestInjector > > > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 10.569 > sec > > > [junit] Running org.apache.nutch.crawl.TestLinkDbMerger > > > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 4.033 > sec > > > [junit] Running org.apache.nutch.crawl.TestSignatureFactory > > > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.491 > sec > > > [junit] Running org.apache.nutch.fetcher.TestFetcher > > > [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 21.039 > sec > > > [junit] Running org.apache.nutch.indexer.TestIndexingFilters > > > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.025 > sec > > > [junit] Running org.apache.nutch.metadata.TestMetadata > > > [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: 0.198 > sec > > > [junit] Running org.apache.nutch.metadata.TestSpellCheckedMetadata > > > [junit] Tests run: 11, Failures: 0, Errors: 0, Time elapsed: 3.679 > sec > > > [junit] Running org.apache.nutch.net.TestURLFilters > > > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.617 > sec > > > [junit] Running org.apache.nutch.net.TestURLNormalizers > > > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.655 > sec > > > [junit] Running org.apache.nutch.parse.TestOutlinkExtractor > > > [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.06 sec > > > [junit] Running org.apache.nutch.parse.TestParseData > > > [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.052 > sec > > > [junit] Running org.apache.nutch.parse.TestParseText > > > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.15 sec > > > [junit] Running org.apache.nutch.parse.TestParserFactory > > > [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 1.077 > sec > > > [junit] Running org.apache.nutch.plugin.TestPluginSystem > > > [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 1.485 > sec > > > [junit] Running org.apache.nutch.protocol.TestContent > > > [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.548 > sec > > > [junit] Running org.apache.nutch.protocol.TestProtocolFactory > > > [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 1.634 > sec > > > [junit] Running org.apache.nutch.segment.TestSegmentMerger > > > [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 135.865 > > > sec [junit] Test org.apache.nutch.segment.TestSegmentMerger FAILED > > > [junit] Running org.apache.nutch.util.TestEncodingDetector > > > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.572 > sec > > > [junit] Running org.apache.nutch.util.TestGZIPUtils > > > [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 1.343 > sec > > > [junit] Running org.apache.nutch.util.TestNodeWalker > > > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.283 > sec > > > [junit] Running org.apache.nutch.util.TestPrefixStringMatcher > > > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.109 > sec > > > [junit] Running org.apache.nutch.util.TestStringUtil > > > [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.055 > sec > > > [junit] Running org.apache.nutch.util.TestSuffixStringMatcher > > > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.125 > sec > > > [junit] Running org.apache.nutch.util.TestURLUtil > > > [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 0.374 > sec > > > > > > BUILD FAILED > > > > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trun > > > k/trunk/build.xml:395: Tests failed! > > > > > > Total time: 5 minutes 20 seconds > > > Build step 'Execute shell' marked build as failure > > > Recording test results > > > Publishing Javadoc > > > Updating NUTCH-1192 > > -- > Markus Jelsma - CTO - Openindex > http://www.linkedin.com/in/markus17 > 050-8536620 / 06-50258350 > -- *Lewis*
