Hi,
If you look at the recently failing Nutch trunk builds, namely #1645,
#1650 & #1651, a common denominator is the
org.apache.nutch.segment.TestSegmentMerger.testLargeMerge
<https://builds.apache.org/job/Nutch-trunk/1651/testReport/org.apache.nutch.segment/TestSegmentMerger/testLargeMerge/>
which failes with the following:
2011-11-03 04:17:37,478 WARN mapred.LocalJobRunner
(LocalJobRunner.java:run(256)) - job_local_0001
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find
taskTracker/jobcache/job_local_0001/attempt_local_0001_m_000000_0/output/file.out
in any of the configured local directories
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:389)
at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:138)
at
org.apache.hadoop.mapred.MapOutputFile.getOutputFile(MapOutputFile.java:50)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:193)
Any clues as to this pattern of failures? Is it possible to nail it
down to the Solaris slave build environment?
On Thu, Nov 3, 2011 at 4:18 AM, Apache Jenkins Server
<[email protected] <mailto:[email protected]>> wrote:
See <https://builds.apache.org/job/Nutch-trunk/1651/changes>
Changes:
[lewismc] commit to make trivial change to working directory
configuration for ant sonar target...
[lewismc] commit to add MySQL driver classpath to sonar analysis
target
[ferdy] NUTCH-1192 Add '/runtime' to svn ignore
------------------------------------------
[...truncated 2838 lines...]
compile:
[echo] Compiling plugin: urlfilter-validator
[javac]
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/build-plugin.xml:117:
warning: 'includeantruntime' was not set, defaulting to
build.sysclasspath=last; set to false for repeatable builds
[javac] Compiling 1 source file to
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlfilter-validator/classes
jar:
[jar] Building jar:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlfilter-validator/urlfilter-validator.jar
deps-test:
deploy:
[copy] Copying 1 file to
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlfilter-validator
copy-generated-lib:
[copy] Copying 1 file to
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlfilter-validator
init:
[mkdir] Created dir:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlmeta
[mkdir] Created dir:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlmeta/classes
[mkdir] Created dir:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlmeta/test
[mkdir] Created dir:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlmeta
init-plugin:
deps-jar:
clean-lib:
resolve-default:
[ivy:resolve] :: loading settings :: file =
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/ivy/ivysettings.xml
compile:
[echo] Compiling plugin: urlmeta
[javac]
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/build-plugin.xml:117:
warning: 'includeantruntime' was not set, defaulting to
build.sysclasspath=last; set to false for repeatable builds
[javac] Compiling 2 source files to
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlmeta/classes
jar:
[jar] Building jar:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlmeta/urlmeta.jar
deps-test:
deploy:
[copy] Copying 1 file to
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlmeta
copy-generated-lib:
[copy] Copying 1 file to
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlmeta
init:
[mkdir] Created dir:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-basic
[mkdir] Created dir:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-basic/classes
[mkdir] Created dir:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-basic/test
[mkdir] Created dir:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-basic
init-plugin:
deps-jar:
clean-lib:
resolve-default:
[ivy:resolve] :: loading settings :: file =
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/ivy/ivysettings.xml
compile:
[echo] Compiling plugin: urlnormalizer-basic
[javac]
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/build-plugin.xml:117:
warning: 'includeantruntime' was not set, defaulting to
build.sysclasspath=last; set to false for repeatable builds
[javac] Compiling 1 source file to
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-basic/classes
jar:
[jar] Building jar:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-basic/urlnormalizer-basic.jar
deps-test:
deploy:
[copy] Copying 1 file to
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-basic
copy-generated-lib:
[copy] Copying 1 file to
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-basic
init:
[mkdir] Created dir:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-pass
[mkdir] Created dir:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-pass/classes
[mkdir] Created dir:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-pass/test
[mkdir] Created dir:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-pass
init-plugin:
deps-jar:
clean-lib:
resolve-default:
[ivy:resolve] :: loading settings :: file =
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/ivy/ivysettings.xml
compile:
[echo] Compiling plugin: urlnormalizer-pass
[javac]
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/build-plugin.xml:117:
warning: 'includeantruntime' was not set, defaulting to
build.sysclasspath=last; set to false for repeatable builds
[javac] Compiling 1 source file to
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-pass/classes
jar:
[jar] Building jar:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-pass/urlnormalizer-pass.jar
deps-test:
deploy:
[copy] Copying 1 file to
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-pass
copy-generated-lib:
[copy] Copying 1 file to
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-pass
[mkdir] Created dir:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-regex/test/data
[copy] Copying 4 files to
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-regex/test/data
init:
[mkdir] Created dir:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-regex/classes
[mkdir] Created dir:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-regex
init-plugin:
deps-jar:
clean-lib:
resolve-default:
[ivy:resolve] :: loading settings :: file =
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/ivy/ivysettings.xml
compile:
[echo] Compiling plugin: urlnormalizer-regex
[javac]
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/build-plugin.xml:117:
warning: 'includeantruntime' was not set, defaulting to
build.sysclasspath=last; set to false for repeatable builds
[javac] Compiling 1 source file to
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-regex/classes
[javac] Note:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/urlnormalizer-regex/src/java/org/apache/nutch/net/urlnormalizer/regex/RegexURLNormalizer.java
uses unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.
jar:
[jar] Building jar:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-regex/urlnormalizer-regex.jar
deps-test:
init:
init-plugin:
clean-lib:
resolve-default:
[ivy:resolve] :: loading settings :: file =
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/ivy/ivysettings.xml
compile:
jar:
deps-test:
deploy:
copy-generated-lib:
deploy:
[copy] Copying 1 file to
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-regex
copy-generated-lib:
[copy] Copying 1 file to
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-regex
compile:
job:
[jar] Building jar:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/nutch-2011-11-03_04-11-51.job
resolve-test:
[taskdef] Could not load definitions from resource
org/sonar/ant/antlib.xml. It could not be found.
copy-libs:
compile-core-test:
[javac]
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build.xml:320:
warning: 'includeantruntime' was not set, defaulting to
build.sysclasspath=last; set to false for repeatable builds
[javac] Compiling 32 source files to
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/test/classes
[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.
test-core:
[mkdir] Created dir:
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/test/data
[copy] Copying 90 files to
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/test/data
[copy] Copying 1 file to
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/test/classes
[copy] Copying 1 file to
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/test/classes
[junit] WARNING: multiple versions of ant detected in path for
junit
[junit]
jar:file:/export/home/hudson/tools/ant/latest/lib/ant.jar!/org/apache/tools/ant/Project.class
[junit] and
jar:file:/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/lib/ant-1.6.5.jar!/org/apache/tools/ant/Project.class
[junit] Running org.apache.nutch.crawl.TestCrawlDbMerger
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
7.829 sec
[junit] Running org.apache.nutch.crawl.TestGenerator
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:
60.748 sec
[junit] Running org.apache.nutch.crawl.TestInjector
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
10.569 sec
[junit] Running org.apache.nutch.crawl.TestLinkDbMerger
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
4.033 sec
[junit] Running org.apache.nutch.crawl.TestSignatureFactory
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
0.491 sec
[junit] Running org.apache.nutch.fetcher.TestFetcher
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed:
21.039 sec
[junit] Running org.apache.nutch.indexer.TestIndexingFilters
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
1.025 sec
[junit] Running org.apache.nutch.metadata.TestMetadata
[junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed:
0.198 sec
[junit] Running org.apache.nutch.metadata.TestSpellCheckedMetadata
[junit] Tests run: 11, Failures: 0, Errors: 0, Time elapsed:
3.679 sec
[junit] Running org.apache.nutch.net.TestURLFilters
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
1.617 sec
[junit] Running org.apache.nutch.net.TestURLNormalizers
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
0.655 sec
[junit] Running org.apache.nutch.parse.TestOutlinkExtractor
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:
0.06 sec
[junit] Running org.apache.nutch.parse.TestParseData
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed:
0.052 sec
[junit] Running org.apache.nutch.parse.TestParseText
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
0.15 sec
[junit] Running org.apache.nutch.parse.TestParserFactory
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed:
1.077 sec
[junit] Running org.apache.nutch.plugin.TestPluginSystem
[junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed:
1.485 sec
[junit] Running org.apache.nutch.protocol.TestContent
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed:
0.548 sec
[junit] Running org.apache.nutch.protocol.TestProtocolFactory
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed:
1.634 sec
[junit] Running org.apache.nutch.segment.TestSegmentMerger
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed:
135.865 sec
[junit] Test org.apache.nutch.segment.TestSegmentMerger FAILED
[junit] Running org.apache.nutch.util.TestEncodingDetector
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
0.572 sec
[junit] Running org.apache.nutch.util.TestGZIPUtils
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:
1.343 sec
[junit] Running org.apache.nutch.util.TestNodeWalker
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
0.283 sec
[junit] Running org.apache.nutch.util.TestPrefixStringMatcher
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
0.109 sec
[junit] Running org.apache.nutch.util.TestStringUtil
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed:
0.055 sec
[junit] Running org.apache.nutch.util.TestSuffixStringMatcher
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
0.125 sec
[junit] Running org.apache.nutch.util.TestURLUtil
[junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed:
0.374 sec
BUILD FAILED
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build.xml:395:
Tests failed!
Total time: 5 minutes 20 seconds
Build step 'Execute shell' marked build as failure
Recording test results
Publishing Javadoc
Updating NUTCH-1192
--
/Lewis/
