See <https://builds.apache.org/job/Mahout-Examples-Classify-20News/237/changes>
Changes: [ssc] MAHOUT-1164 Make ARFF integration generate meta-data in JSON format ------------------------------------------ [...truncated 3850 lines...] Jun 9, 2013 12:06:46 PM org.apache.hadoop.mapred.Counters log INFO: Reduce input records=18846 Jun 9, 2013 12:06:46 PM org.apache.hadoop.mapred.Counters log INFO: Reduce input groups=18846 Jun 9, 2013 12:06:46 PM org.apache.hadoop.mapred.Counters log INFO: Combine output records=0 Jun 9, 2013 12:06:46 PM org.apache.hadoop.mapred.Counters log INFO: Physical memory (bytes) snapshot=0 Jun 9, 2013 12:06:46 PM org.apache.hadoop.mapred.Counters log INFO: Reduce output records=18846 Jun 9, 2013 12:06:46 PM org.apache.hadoop.mapred.Counters log INFO: Virtual memory (bytes) snapshot=0 Jun 9, 2013 12:06:46 PM org.apache.hadoop.mapred.Counters log INFO: Map output records=18846 Jun 9, 2013 12:06:46 PM org.slf4j.impl.JCLLoggerAdapter info INFO: Deleting /tmp/mahout-work-jenkins/20news-vectors/tfidf-vectors Jun 9, 2013 12:06:47 PM org.apache.hadoop.mapreduce.lib.input.FileInputFormat listStatus INFO: Total input paths to process : 1 Jun 9, 2013 12:06:47 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob INFO: Running job: job_local_0009 Jun 9, 2013 12:06:47 PM org.apache.hadoop.mapred.Task initialize INFO: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@11415c8 Jun 9, 2013 12:06:47 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init> INFO: io.sort.mb = 100 Jun 9, 2013 12:06:47 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init> INFO: data buffer = 79691776/99614720 Jun 9, 2013 12:06:47 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init> INFO: record buffer = 262144/327680 Jun 9, 2013 12:06:48 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob INFO: map 0% reduce 0% Jun 9, 2013 12:06:49 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer flush INFO: Starting flush of map output Jun 9, 2013 12:06:49 PM org.apache.hadoop.mapred.MapTask$MapOutputBuffer sortAndSpill INFO: Finished spill 0 Jun 9, 2013 12:06:49 PM org.apache.hadoop.mapred.Task done INFO: Task:attempt_local_0009_m_000000_0 is done. And is in the process of commiting Jun 9, 2013 12:06:49 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate INFO: Jun 9, 2013 12:06:49 PM org.apache.hadoop.mapred.Task sendDone INFO: Task 'attempt_local_0009_m_000000_0' done. Jun 9, 2013 12:06:49 PM org.apache.hadoop.mapred.Task initialize INFO: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@8990e4 Jun 9, 2013 12:06:49 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate INFO: Jun 9, 2013 12:06:49 PM org.apache.hadoop.mapred.Merger$MergeQueue merge INFO: Merging 1 sorted segments Jun 9, 2013 12:06:49 PM org.apache.hadoop.mapred.Merger$MergeQueue merge INFO: Down to the last merge-pass, with 1 segments left of total size: 28437746 bytes Jun 9, 2013 12:06:49 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate INFO: Jun 9, 2013 12:06:50 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob INFO: map 100% reduce 0% Jun 9, 2013 12:06:51 PM org.apache.hadoop.mapred.Task done INFO: Task:attempt_local_0009_r_000000_0 is done. And is in the process of commiting Jun 9, 2013 12:06:51 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate INFO: Jun 9, 2013 12:06:51 PM org.apache.hadoop.mapred.Task commit INFO: Task attempt_local_0009_r_000000_0 is allowed to commit now Jun 9, 2013 12:06:51 PM org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter commitTask INFO: Saved output of task 'attempt_local_0009_r_000000_0' to /tmp/mahout-work-jenkins/20news-vectors/tfidf-vectors Jun 9, 2013 12:06:51 PM org.apache.hadoop.mapred.LocalJobRunner$Job statusUpdate INFO: reduce > reduce Jun 9, 2013 12:06:51 PM org.apache.hadoop.mapred.Task sendDone INFO: Task 'attempt_local_0009_r_000000_0' done. Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob INFO: map 100% reduce 100% Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob INFO: Job complete: job_local_0009 Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: Counters: 20 Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: File Output Format Counters Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: Bytes Written=28913427 Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: FileSystemCounters Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: FILE_BYTES_READ=1728241830 Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: FILE_BYTES_WRITTEN=1600909389 Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: File Input Format Counters Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: Bytes Read=28913427 Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: Map-Reduce Framework Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: Map output materialized bytes=28437750 Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: Map input records=18846 Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: Reduce shuffle bytes=0 Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: Spilled Records=37692 Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: Map output bytes=28362505 Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: Total committed heap usage (bytes)=2461270016 Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: CPU time spent (ms)=0 Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: SPLIT_RAW_BYTES=140 Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: Combine input records=0 Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: Reduce input records=18846 Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: Reduce input groups=18846 Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: Combine output records=0 Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: Physical memory (bytes) snapshot=0 Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: Reduce output records=18846 Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: Virtual memory (bytes) snapshot=0 Jun 9, 2013 12:06:52 PM org.apache.hadoop.mapred.Counters log INFO: Map output records=18846 Jun 9, 2013 12:06:52 PM org.slf4j.impl.JCLLoggerAdapter info INFO: Deleting /tmp/mahout-work-jenkins/20news-vectors/partial-vectors-0 Jun 9, 2013 12:06:52 PM org.slf4j.impl.JCLLoggerAdapter info INFO: Program took 72296 ms (Minutes: 1.2049333333333334) + echo 'Creating training and holdout set with a random 80-20 split of the generated vector dataset' Creating training and holdout set with a random 80-20 split of the generated vector dataset + ./bin/mahout split -i /tmp/mahout-work-jenkins/20news-vectors/tfidf-vectors --trainingOutput /tmp/mahout-work-jenkins/20news-train-vectors --testOutput /tmp/mahout-work-jenkins/20news-test-vectors --randomSelectionPct 40 --overwrite --sequenceFiles -xm sequential hadoop binary is not in PATH,HADOOP_HOME/bin,HADOOP_PREFIX/bin, running locally SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Classify-20News/ws/trunk/examples/target/mahout-examples-0.8-SNAPSHOT-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]> SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Classify-20News/ws/trunk/examples/target/dependency/slf4j-jcl-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.JCLLoggerFactory] Jun 9, 2013 12:06:53 PM org.slf4j.impl.JCLLoggerAdapter warn WARNING: No split.props found on classpath, will use command-line arguments only Jun 9, 2013 12:06:54 PM org.slf4j.impl.JCLLoggerAdapter info INFO: Command line arguments: {--endPhase=[2147483647], --input=[/tmp/mahout-work-jenkins/20news-vectors/tfidf-vectors], --method=[sequential], --overwrite=null, --randomSelectionPct=[40], --sequenceFiles=null, --startPhase=[0], --tempDir=[temp], --testOutput=[/tmp/mahout-work-jenkins/20news-test-vectors], --trainingOutput=[/tmp/mahout-work-jenkins/20news-train-vectors]} Jun 9, 2013 12:06:54 PM org.slf4j.impl.JCLLoggerAdapter info INFO: Deleting /tmp/mahout-work-jenkins/20news-train-vectors Jun 9, 2013 12:06:54 PM org.slf4j.impl.JCLLoggerAdapter info INFO: Deleting /tmp/mahout-work-jenkins/20news-test-vectors Jun 9, 2013 12:06:57 PM org.slf4j.impl.JCLLoggerAdapter info INFO: part-r-00000 has 162419 lines Jun 9, 2013 12:06:57 PM org.slf4j.impl.JCLLoggerAdapter info INFO: part-r-00000 test split size is 64968 based on random selection percentage 40 Jun 9, 2013 12:06:58 PM org.apache.hadoop.util.NativeCodeLoader <clinit> WARNING: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Jun 9, 2013 12:06:58 PM org.apache.hadoop.io.compress.CodecPool getCompressor INFO: Got brand-new compressor Jun 9, 2013 12:06:58 PM org.apache.hadoop.io.compress.CodecPool getCompressor INFO: Got brand-new compressor Jun 9, 2013 12:07:08 PM org.slf4j.impl.JCLLoggerAdapter info INFO: file: part-r-00000, input: 162419 train: 11324, test: 7522 starting at 0 Jun 9, 2013 12:07:08 PM org.slf4j.impl.JCLLoggerAdapter info INFO: Program took 14370 ms (Minutes: 0.2395) + echo 'Training Naive Bayes model' Training Naive Bayes model + ./bin/mahout trainnb -i /tmp/mahout-work-jenkins/20news-train-vectors -el -o /tmp/mahout-work-jenkins/model -li /tmp/mahout-work-jenkins/labelindex -ow hadoop binary is not in PATH,HADOOP_HOME/bin,HADOOP_PREFIX/bin, running locally SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Classify-20News/ws/trunk/examples/target/mahout-examples-0.8-SNAPSHOT-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]> SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Classify-20News/ws/trunk/examples/target/dependency/slf4j-jcl-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.JCLLoggerFactory] Jun 9, 2013 12:07:09 PM org.slf4j.impl.JCLLoggerAdapter warn WARNING: No trainnb.props found on classpath, will use command-line arguments only Jun 9, 2013 12:07:10 PM org.slf4j.impl.JCLLoggerAdapter info INFO: Command line arguments: {--alphaI=[1.0], --endPhase=[2147483647], --extractLabels=null, --input=[/tmp/mahout-work-jenkins/20news-train-vectors], --labelIndex=[/tmp/mahout-work-jenkins/labelindex], --output=[/tmp/mahout-work-jenkins/model], --overwrite=null, --startPhase=[0], --tempDir=[temp]} Jun 9, 2013 12:07:10 PM org.slf4j.impl.JCLLoggerAdapter info INFO: Deleting /tmp/mahout-work-jenkins/model Jun 9, 2013 12:07:10 PM org.apache.hadoop.util.NativeCodeLoader <clinit> WARNING: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Jun 9, 2013 12:07:11 PM org.apache.hadoop.io.compress.CodecPool getDecompressor INFO: Got brand-new decompressor Jun 9, 2013 12:07:18 PM org.apache.hadoop.mapreduce.lib.input.FileInputFormat listStatus INFO: Total input paths to process : 1 Jun 9, 2013 12:07:19 PM org.apache.hadoop.filecache.TrackerDistributedCacheManager downloadCacheObject INFO: Creating labelindex in /tmp/hadoop-jenkins/mapred/local/archive/4772107937303584349_-535727449_685070214/file/tmp/mahout-work-jenkins-work-4940216929170236274 with rwxr-xr-x Jun 9, 2013 12:07:19 PM org.apache.hadoop.filecache.TrackerDistributedCacheManager downloadCacheObject INFO: Cached /tmp/mahout-work-jenkins/labelindex as /tmp/hadoop-jenkins/mapred/local/archive/4772107937303584349_-535727449_685070214/file/tmp/mahout-work-jenkins/labelindex Jun 9, 2013 12:07:19 PM org.apache.hadoop.filecache.TrackerDistributedCacheManager localizePublicCacheObject INFO: Cached /tmp/mahout-work-jenkins/labelindex as /tmp/hadoop-jenkins/mapred/local/archive/4772107937303584349_-535727449_685070214/file/tmp/mahout-work-jenkins/labelindex Jun 9, 2013 12:07:19 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob INFO: Running job: job_local_0001 Jun 9, 2013 12:07:19 PM org.apache.hadoop.util.ProcessTree isSetsidSupported INFO: setsid exited with exit code 0 Jun 9, 2013 12:07:19 PM org.apache.hadoop.mapred.Task initialize INFO: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@8c5ea2 Jun 9, 2013 12:07:19 PM org.apache.hadoop.mapred.LocalJobRunner$Job run WARNING: job_local_0001 java.lang.ClassCastException: org.apache.hadoop.mapreduce.lib.input.FileSplit cannot be cast to org.apache.hadoop.mapred.InputSplit at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:412) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:214) Jun 9, 2013 12:07:20 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob INFO: map 0% reduce 0% Jun 9, 2013 12:07:20 PM org.apache.hadoop.mapred.JobClient monitorAndPrintJob INFO: Job complete: job_local_0001 Jun 9, 2013 12:07:20 PM org.apache.hadoop.mapred.Counters log INFO: Counters: 0 Jun 9, 2013 12:07:20 PM org.slf4j.impl.JCLLoggerAdapter info INFO: Program took 10642 ms (Minutes: 0.17736666666666667) + echo 'Self testing on training set' Self testing on training set + ./bin/mahout testnb -i /tmp/mahout-work-jenkins/20news-train-vectors -m /tmp/mahout-work-jenkins/model -l /tmp/mahout-work-jenkins/labelindex -ow -o /tmp/mahout-work-jenkins/20news-testing hadoop binary is not in PATH,HADOOP_HOME/bin,HADOOP_PREFIX/bin, running locally SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Classify-20News/ws/trunk/examples/target/mahout-examples-0.8-SNAPSHOT-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]> SLF4J: Found binding in [jar:<https://builds.apache.org/job/Mahout-Examples-Classify-20News/ws/trunk/examples/target/dependency/slf4j-jcl-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.JCLLoggerFactory] Jun 9, 2013 12:07:22 PM org.slf4j.impl.JCLLoggerAdapter warn WARNING: No testnb.props found on classpath, will use command-line arguments only Jun 9, 2013 12:07:22 PM org.slf4j.impl.JCLLoggerAdapter info INFO: Command line arguments: {--endPhase=[2147483647], --input=[/tmp/mahout-work-jenkins/20news-train-vectors], --labelIndex=[/tmp/mahout-work-jenkins/labelindex], --model=[/tmp/mahout-work-jenkins/model], --output=[/tmp/mahout-work-jenkins/20news-testing], --overwrite=null, --startPhase=[0], --tempDir=[temp]} Jun 9, 2013 12:07:23 PM org.slf4j.impl.JCLLoggerAdapter info INFO: Deleting /tmp/mahout-work-jenkins/20news-testing Jun 9, 2013 12:07:23 PM org.apache.hadoop.util.NativeCodeLoader <clinit> WARNING: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Jun 9, 2013 12:07:23 PM org.apache.hadoop.mapred.JobClient$2 run INFO: Cleaning up the staging area file:/tmp/hadoop-jenkins/mapred/staging/jenkins592238449/.staging/job_local_0001 Jun 9, 2013 12:07:23 PM org.apache.hadoop.security.UserGroupInformation doAs SEVERE: PriviledgedActionException as:jenkins cause:java.io.FileNotFoundException: File /tmp/mahout-work-jenkins/model does not exist. Exception in thread "main" java.io.FileNotFoundException: File /tmp/mahout-work-jenkins/model does not exist. at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:397) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251) at org.apache.hadoop.filecache.DistributedCache.getFileStatus(DistributedCache.java:185) at org.apache.hadoop.filecache.TrackerDistributedCacheManager.determineTimestamps(TrackerDistributedCacheManager.java:732) at org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:825) at org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:717) at org.apache.hadoop.mapred.JobClient.access$400(JobClient.java:179) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:927) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149) at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912) at org.apache.hadoop.mapreduce.Job.submit(Job.java:500) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530) at org.apache.mahout.classifier.naivebayes.test.TestNaiveBayesDriver.runMapReduce(TestNaiveBayesDriver.java:141) at org.apache.mahout.classifier.naivebayes.test.TestNaiveBayesDriver.run(TestNaiveBayesDriver.java:109) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.mahout.classifier.naivebayes.test.TestNaiveBayesDriver.main(TestNaiveBayesDriver.java:66) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68) at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139) at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195) Build step 'Execute shell' marked build as failure
