Hi,
If you look at the recently failing Nutch trunk builds, namely #1645, #1650
& #1651, a common denominator is the
org.apache.nutch.segment.TestSegmentMerger.testLargeMerge<https://builds.apache.org/job/Nutch-trunk/1651/testReport/org.apache.nutch.segment/TestSegmentMerger/testLargeMerge/>
which failes with the following:
2011-11-03 04:17:37,478 WARN mapred.LocalJobRunner
(LocalJobRunner.java:run(256)) - job_local_0001
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find
taskTracker/jobcache/job_local_0001/attempt_local_0001_m_000000_0/output/file.out
in any of the configured local directories
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:389)
at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:138)
at
org.apache.hadoop.mapred.MapOutputFile.getOutputFile(MapOutputFile.java:50)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:193)
Any clues as to this pattern of failures? Is it possible to nail it down to
the Solaris slave build environment?
On Thu, Nov 3, 2011 at 4:18 AM, Apache Jenkins Server <
[email protected]> wrote:
> See <https://builds.apache.org/job/Nutch-trunk/1651/changes>
>
> Changes:
>
> [lewismc] commit to make trivial change to working directory configuration
> for ant sonar target...
>
> [lewismc] commit to add MySQL driver classpath to sonar analysis target
>
> [ferdy] NUTCH-1192 Add '/runtime' to svn ignore
>
> ------------------------------------------
> [...truncated 2838 lines...]
> compile:
> [echo] Compiling plugin: urlfilter-validator
> [javac]
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/build-plugin.xml:117:
> warning: 'includeantruntime' was not set, defaulting to
> build.sysclasspath=last; set to false for repeatable builds
> [javac] Compiling 1 source file to
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlfilter-validator/classes
>
> jar:
> [jar] Building jar:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlfilter-validator/urlfilter-validator.jar
>
> deps-test:
>
> deploy:
> [copy] Copying 1 file to
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlfilter-validator
>
> copy-generated-lib:
> [copy] Copying 1 file to
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlfilter-validator
>
> init:
> [mkdir] Created dir:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlmeta
> [mkdir] Created dir:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlmeta/classes
> [mkdir] Created dir:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlmeta/test
> [mkdir] Created dir:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlmeta
>
> init-plugin:
>
> deps-jar:
>
> clean-lib:
>
> resolve-default:
> [ivy:resolve] :: loading settings :: file =
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/ivy/ivysettings.xml
>
> compile:
> [echo] Compiling plugin: urlmeta
> [javac]
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/build-plugin.xml:117:
> warning: 'includeantruntime' was not set, defaulting to
> build.sysclasspath=last; set to false for repeatable builds
> [javac] Compiling 2 source files to
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlmeta/classes
>
> jar:
> [jar] Building jar:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlmeta/urlmeta.jar
>
> deps-test:
>
> deploy:
> [copy] Copying 1 file to
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlmeta
>
> copy-generated-lib:
> [copy] Copying 1 file to
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlmeta
>
> init:
> [mkdir] Created dir:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-basic
> [mkdir] Created dir:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-basic/classes
> [mkdir] Created dir:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-basic/test
> [mkdir] Created dir:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-basic
>
> init-plugin:
>
> deps-jar:
>
> clean-lib:
>
> resolve-default:
> [ivy:resolve] :: loading settings :: file =
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/ivy/ivysettings.xml
>
> compile:
> [echo] Compiling plugin: urlnormalizer-basic
> [javac]
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/build-plugin.xml:117:
> warning: 'includeantruntime' was not set, defaulting to
> build.sysclasspath=last; set to false for repeatable builds
> [javac] Compiling 1 source file to
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-basic/classes
>
> jar:
> [jar] Building jar:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-basic/urlnormalizer-basic.jar
>
> deps-test:
>
> deploy:
> [copy] Copying 1 file to
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-basic
>
> copy-generated-lib:
> [copy] Copying 1 file to
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-basic
>
> init:
> [mkdir] Created dir:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-pass
> [mkdir] Created dir:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-pass/classes
> [mkdir] Created dir:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-pass/test
> [mkdir] Created dir:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-pass
>
> init-plugin:
>
> deps-jar:
>
> clean-lib:
>
> resolve-default:
> [ivy:resolve] :: loading settings :: file =
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/ivy/ivysettings.xml
>
> compile:
> [echo] Compiling plugin: urlnormalizer-pass
> [javac]
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/build-plugin.xml:117:
> warning: 'includeantruntime' was not set, defaulting to
> build.sysclasspath=last; set to false for repeatable builds
> [javac] Compiling 1 source file to
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-pass/classes
>
> jar:
> [jar] Building jar:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-pass/urlnormalizer-pass.jar
>
> deps-test:
>
> deploy:
> [copy] Copying 1 file to
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-pass
>
> copy-generated-lib:
> [copy] Copying 1 file to
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-pass
> [mkdir] Created dir:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-regex/test/data
> [copy] Copying 4 files to
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-regex/test/data
>
> init:
> [mkdir] Created dir:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-regex/classes
> [mkdir] Created dir:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-regex
>
> init-plugin:
>
> deps-jar:
>
> clean-lib:
>
> resolve-default:
> [ivy:resolve] :: loading settings :: file =
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/ivy/ivysettings.xml
>
> compile:
> [echo] Compiling plugin: urlnormalizer-regex
> [javac]
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/build-plugin.xml:117:
> warning: 'includeantruntime' was not set, defaulting to
> build.sysclasspath=last; set to false for repeatable builds
> [javac] Compiling 1 source file to
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-regex/classes
> [javac] Note:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/urlnormalizer-regex/src/java/org/apache/nutch/net/urlnormalizer/regex/RegexURLNormalizer.java
> uses unchecked or unsafe operations.
> [javac] Note: Recompile with -Xlint:unchecked for details.
>
> jar:
> [jar] Building jar:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-regex/urlnormalizer-regex.jar
>
> deps-test:
>
> init:
>
> init-plugin:
>
> clean-lib:
>
> resolve-default:
> [ivy:resolve] :: loading settings :: file =
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/ivy/ivysettings.xml
>
> compile:
>
> jar:
>
> deps-test:
>
> deploy:
>
> copy-generated-lib:
>
> deploy:
> [copy] Copying 1 file to
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-regex
>
> copy-generated-lib:
> [copy] Copying 1 file to
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-regex
>
> compile:
>
> job:
> [jar] Building jar:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/nutch-2011-11-03_04-11-51.job
>
> resolve-test:
> [taskdef] Could not load definitions from resource
> org/sonar/ant/antlib.xml. It could not be found.
>
> copy-libs:
>
> compile-core-test:
> [javac]
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build.xml:320:
> warning: 'includeantruntime' was not set, defaulting to
> build.sysclasspath=last; set to false for repeatable builds
> [javac] Compiling 32 source files to
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/test/classes
> [javac] Note: Some input files use or override a deprecated API.
> [javac] Note: Recompile with -Xlint:deprecation for details.
> [javac] Note: Some input files use unchecked or unsafe operations.
> [javac] Note: Recompile with -Xlint:unchecked for details.
>
> test-core:
> [mkdir] Created dir:
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/test/data
> [copy] Copying 90 files to
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/test/data
> [copy] Copying 1 file to
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/test/classes
> [copy] Copying 1 file to
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/test/classes
> [junit] WARNING: multiple versions of ant detected in path for junit
> [junit]
>
> jar:file:/export/home/hudson/tools/ant/latest/lib/ant.jar!/org/apache/tools/ant/Project.class
> [junit] and
> jar:file:/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/lib/ant-1.6.5.jar!/org/apache/tools/ant/Project.class
> [junit] Running org.apache.nutch.crawl.TestCrawlDbMerger
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 7.829 sec
> [junit] Running org.apache.nutch.crawl.TestGenerator
> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 60.748 sec
> [junit] Running org.apache.nutch.crawl.TestInjector
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 10.569 sec
> [junit] Running org.apache.nutch.crawl.TestLinkDbMerger
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 4.033 sec
> [junit] Running org.apache.nutch.crawl.TestSignatureFactory
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.491 sec
> [junit] Running org.apache.nutch.fetcher.TestFetcher
> [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 21.039 sec
> [junit] Running org.apache.nutch.indexer.TestIndexingFilters
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.025 sec
> [junit] Running org.apache.nutch.metadata.TestMetadata
> [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: 0.198 sec
> [junit] Running org.apache.nutch.metadata.TestSpellCheckedMetadata
> [junit] Tests run: 11, Failures: 0, Errors: 0, Time elapsed: 3.679 sec
> [junit] Running org.apache.nutch.net.TestURLFilters
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.617 sec
> [junit] Running org.apache.nutch.net.TestURLNormalizers
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.655 sec
> [junit] Running org.apache.nutch.parse.TestOutlinkExtractor
> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.06 sec
> [junit] Running org.apache.nutch.parse.TestParseData
> [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.052 sec
> [junit] Running org.apache.nutch.parse.TestParseText
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.15 sec
> [junit] Running org.apache.nutch.parse.TestParserFactory
> [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 1.077 sec
> [junit] Running org.apache.nutch.plugin.TestPluginSystem
> [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 1.485 sec
> [junit] Running org.apache.nutch.protocol.TestContent
> [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.548 sec
> [junit] Running org.apache.nutch.protocol.TestProtocolFactory
> [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 1.634 sec
> [junit] Running org.apache.nutch.segment.TestSegmentMerger
> [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 135.865 sec
> [junit] Test org.apache.nutch.segment.TestSegmentMerger FAILED
> [junit] Running org.apache.nutch.util.TestEncodingDetector
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.572 sec
> [junit] Running org.apache.nutch.util.TestGZIPUtils
> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 1.343 sec
> [junit] Running org.apache.nutch.util.TestNodeWalker
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.283 sec
> [junit] Running org.apache.nutch.util.TestPrefixStringMatcher
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.109 sec
> [junit] Running org.apache.nutch.util.TestStringUtil
> [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.055 sec
> [junit] Running org.apache.nutch.util.TestSuffixStringMatcher
> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.125 sec
> [junit] Running org.apache.nutch.util.TestURLUtil
> [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 0.374 sec
>
> BUILD FAILED
> /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build.xml:395:
> Tests failed!
>
> Total time: 5 minutes 20 seconds
> Build step 'Execute shell' marked build as failure
> Recording test results
> Publishing Javadoc
> Updating NUTCH-1192
>
>
--
*Lewis*
