Hi Guys, Our trunk builds have been broken since migrating to new Hadoop 0.20.2 and migrating CrawlDBScanner to new MR API e.g. trunk build [1] 1698. Looking to the stack trace, I'm assuming that this has to do with how we are specifying the new file reads. Hopefully this shouldn't be too hard to solve so maybe we can get on to it at some stage in the near future.
I just want to say Merry Christmas to EVERYONE celebrating and happy holidays to everyone else who may not be. Best Lewis [1] https://builds.apache.org/view/M-R/view/Nutch/job/Nutch-trunk/1698/ On Sat, Dec 24, 2011 at 7:36 AM, Apache Jenkins Server <[email protected]> wrote: > See <https://builds.apache.org/job/Nutch-trunk/1702/changes> > > Changes: > > [markus] Updated pom to reflect Hadoop upgrade > > ------------------------------------------ > [...truncated 2836 lines...] > resolve-default: > [ivy:resolve] :: loading settings :: file = > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/ivy/ivysettings.xml > > compile: > [echo] Compiling plugin: urlfilter-validator > [javac] > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/build-plugin.xml:117: > warning: 'includeantruntime' was not set, defaulting to > build.sysclasspath=last; set to false for repeatable builds > [javac] Compiling 1 source file to > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlfilter-validator/classes > > jar: > [jar] Building jar: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlfilter-validator/urlfilter-validator.jar > > deps-test: > > deploy: > [copy] Copying 1 file to > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlfilter-validator > > copy-generated-lib: > [copy] Copying 1 file to > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlfilter-validator > > init: > [mkdir] Created dir: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlmeta > [mkdir] Created dir: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlmeta/classes > [mkdir] Created dir: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlmeta/test > [mkdir] Created dir: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlmeta > > init-plugin: > > deps-jar: > > clean-lib: > > resolve-default: > [ivy:resolve] :: loading settings :: file = > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/ivy/ivysettings.xml > > compile: > [echo] Compiling plugin: urlmeta > [javac] > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/build-plugin.xml:117: > warning: 'includeantruntime' was not set, defaulting to > build.sysclasspath=last; set to false for repeatable builds > [javac] Compiling 2 source files to > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlmeta/classes > > jar: > [jar] Building jar: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlmeta/urlmeta.jar > > deps-test: > > deploy: > [copy] Copying 1 file to > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlmeta > > copy-generated-lib: > [copy] Copying 1 file to > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlmeta > > init: > [mkdir] Created dir: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-basic > [mkdir] Created dir: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-basic/classes > [mkdir] Created dir: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-basic/test > [mkdir] Created dir: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-basic > > init-plugin: > > deps-jar: > > clean-lib: > > resolve-default: > [ivy:resolve] :: loading settings :: file = > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/ivy/ivysettings.xml > > compile: > [echo] Compiling plugin: urlnormalizer-basic > [javac] > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/build-plugin.xml:117: > warning: 'includeantruntime' was not set, defaulting to > build.sysclasspath=last; set to false for repeatable builds > [javac] Compiling 1 source file to > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-basic/classes > > jar: > [jar] Building jar: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-basic/urlnormalizer-basic.jar > > deps-test: > > deploy: > [copy] Copying 1 file to > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-basic > > copy-generated-lib: > [copy] Copying 1 file to > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-basic > > init: > [mkdir] Created dir: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-pass > [mkdir] Created dir: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-pass/classes > [mkdir] Created dir: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-pass/test > [mkdir] Created dir: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-pass > > init-plugin: > > deps-jar: > > clean-lib: > > resolve-default: > [ivy:resolve] :: loading settings :: file = > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/ivy/ivysettings.xml > > compile: > [echo] Compiling plugin: urlnormalizer-pass > [javac] > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/build-plugin.xml:117: > warning: 'includeantruntime' was not set, defaulting to > build.sysclasspath=last; set to false for repeatable builds > [javac] Compiling 1 source file to > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-pass/classes > > jar: > [jar] Building jar: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-pass/urlnormalizer-pass.jar > > deps-test: > > deploy: > [copy] Copying 1 file to > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-pass > > copy-generated-lib: > [copy] Copying 1 file to > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-pass > [mkdir] Created dir: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-regex/test/data > [copy] Copying 4 files to > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-regex/test/data > > init: > [mkdir] Created dir: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-regex/classes > [mkdir] Created dir: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-regex > > init-plugin: > > deps-jar: > > clean-lib: > > resolve-default: > [ivy:resolve] :: loading settings :: file = > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/ivy/ivysettings.xml > > compile: > [echo] Compiling plugin: urlnormalizer-regex > [javac] > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/build-plugin.xml:117: > warning: 'includeantruntime' was not set, defaulting to > build.sysclasspath=last; set to false for repeatable builds > [javac] Compiling 1 source file to > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-regex/classes > [javac] Note: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/urlnormalizer-regex/src/java/org/apache/nutch/net/urlnormalizer/regex/RegexURLNormalizer.java > uses unchecked or unsafe operations. > [javac] Note: Recompile with -Xlint:unchecked for details. > > jar: > [jar] Building jar: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-regex/urlnormalizer-regex.jar > > deps-test: > > init: > > init-plugin: > > clean-lib: > > resolve-default: > [ivy:resolve] :: loading settings :: file = > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/ivy/ivysettings.xml > > compile: > > jar: > > deps-test: > > deploy: > > copy-generated-lib: > > deploy: > [copy] Copying 1 file to > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-regex > > copy-generated-lib: > [copy] Copying 1 file to > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-regex > > compile: > > job: > [jar] Building jar: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/nutch-2011-12-24_07-29-25.job > > resolve-test: > [taskdef] Could not load definitions from resource org/sonar/ant/antlib.xml. > It could not be found. > > copy-libs: > > compile-core-test: > [javac] > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build.xml:320: > warning: 'includeantruntime' was not set, defaulting to > build.sysclasspath=last; set to false for repeatable builds > [javac] Compiling 32 source files to > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/test/classes > [javac] Note: Some input files use or override a deprecated API. > [javac] Note: Recompile with -Xlint:deprecation for details. > [javac] Note: Some input files use unchecked or unsafe operations. > [javac] Note: Recompile with -Xlint:unchecked for details. > > test-core: > [mkdir] Created dir: > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/test/data > [copy] Copying 90 files to > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/test/data > [copy] Copying 1 file to > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/test/classes > [copy] Copying 1 file to > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/test/classes > [junit] Running org.apache.nutch.crawl.TestCrawlDbMerger > [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 5.721 sec > [junit] Test org.apache.nutch.crawl.TestCrawlDbMerger FAILED > [junit] Running org.apache.nutch.crawl.TestGenerator > [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 59.082 sec > [junit] Running org.apache.nutch.crawl.TestInjector > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 13.243 sec > [junit] Running org.apache.nutch.crawl.TestLinkDbMerger > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 4.266 sec > [junit] Running org.apache.nutch.crawl.TestSignatureFactory > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.449 sec > [junit] Running org.apache.nutch.fetcher.TestFetcher > [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 21.793 sec > [junit] Running org.apache.nutch.indexer.TestIndexingFilters > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.133 sec > [junit] Running org.apache.nutch.metadata.TestMetadata > [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: 0.067 sec > [junit] Running org.apache.nutch.metadata.TestSpellCheckedMetadata > [junit] Tests run: 11, Failures: 0, Errors: 0, Time elapsed: 3.342 sec > [junit] Running org.apache.nutch.net.TestURLFilters > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.406 sec > [junit] Running org.apache.nutch.net.TestURLNormalizers > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.643 sec > [junit] Running org.apache.nutch.parse.TestOutlinkExtractor > [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.062 sec > [junit] Running org.apache.nutch.parse.TestParseData > [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.044 sec > [junit] Running org.apache.nutch.parse.TestParseText > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.068 sec > [junit] Running org.apache.nutch.parse.TestParserFactory > [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 1.014 sec > [junit] Running org.apache.nutch.plugin.TestPluginSystem > [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 1.523 sec > [junit] Running org.apache.nutch.protocol.TestContent > [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.565 sec > [junit] Running org.apache.nutch.protocol.TestProtocolFactory > [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.889 sec > [junit] Running org.apache.nutch.segment.TestSegmentMerger > [junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 184.385 sec > [junit] Test org.apache.nutch.segment.TestSegmentMerger FAILED > [junit] Running org.apache.nutch.util.TestEncodingDetector > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 4.555 sec > [junit] Running org.apache.nutch.util.TestGZIPUtils > [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 1.107 sec > [junit] Running org.apache.nutch.util.TestNodeWalker > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.906 sec > [junit] Running org.apache.nutch.util.TestPrefixStringMatcher > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.14 sec > [junit] Running org.apache.nutch.util.TestStringUtil > [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.056 sec > [junit] Running org.apache.nutch.util.TestSuffixStringMatcher > [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.108 sec > [junit] Running org.apache.nutch.util.TestURLUtil > [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 0.412 sec > > BUILD FAILED > /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build.xml:395: > Tests failed! > > Total time: 6 minutes 22 seconds > Build step 'Execute shell' marked build as failure > Recording test results > Publishing Javadoc > -- Lewis
