Well, the exception doesn't mention it, but i would guess it has something to do with permissions.
-----Original message----- > From:DB Design <[email protected]> > Sent: Tuesday 22nd August 2017 19:33 > To: [email protected] > Subject: run nutch from tomcat with ProcessBuilder > > Hi, > i want to run nutch crawler command from tomcat web application with java > ProcessBuilder, when i run crawler command from terminal every thing is ok, > but when run with ProccessBuilder job fails with below error. > nutch version: 1.12 > java version: 8 > OS: ubuntu 16.04 > tomcat version: 8 > solr version: 6.2 > thanks for your help. > java.lang.Exception: java.io.IOException: Mkdirs failed to create > file:/generate-temp-8dca91a6-3610-4802-a534-c0cdf85cde73/_temporary/0/_temporary/attempt_local1345668275_0001_r_000001_0/fetchlist-1 > at > org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462) > at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:529) > Caused by: java.io.IOException: Mkdirs failed to create > file:/generate-temp-8dca91a6-3610-4802-a534-c0cdf85cde73/_temporary/0/_temporary/attempt_local1345668275_0001_r_000001_0/fetchlist-1 > at > org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:438) > at > org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:424) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:906) > at > org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:1071) > at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:270) > at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:527) > at > org.apache.hadoop.mapred.SequenceFileOutputFormat.getRecordWriter(SequenceFileOutputFormat.java:63) > at > org.apache.hadoop.mapred.lib.MultipleSequenceFileOutputFormat.getBaseRecordWriter(MultipleSequenceFileOutputFormat.java:51) > at > org.apache.hadoop.mapred.lib.MultipleOutputFormat$1.write(MultipleOutputFormat.java:104) > at > org.apache.hadoop.mapred.ReduceTask$OldTrackingRecordWriter.write(ReduceTask.java:493) > at org.apache.hadoop.mapred.ReduceTask$3.collect(ReduceTask.java:422) > at org.apache.nutch.crawl.Generator$Selector.reduce(Generator.java:342) > at org.apache.nutch.crawl.Generator$Selector.reduce(Generator.java:110) > at > org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:444) > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392) > at > org.apache.hadoop.mapred.LocalJobRunner$Job$ReduceTaskRunnable.run(LocalJobRunner.java:319) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > 2017-08-22 20:58:28,674 INFO mapreduce.Job - Job job_local1345668275_0001 > running in uber mode : false > 2017-08-22 20:58:28,674 INFO mapreduce.Job - map 100% reduce 50% > 2017-08-22 20:58:28,675 INFO mapreduce.Job - Job job_local1345668275_0001 > failed with state FAILED due to: NA > 2017-08-22 20:58:28,684 INFO mapreduce.Job - Counters: 33 > File System Counters > FILE: Number of bytes read=1601520 > FILE: Number of bytes written=2209492 > FILE: Number of read operations=0 > FILE: Number of large read operations=0 > FILE: Number of write operations=0 > Map-Reduce Framework > Map input records=1 > Map output records=1 > Map output bytes=77 > Map output materialized bytes=88 > Input split bytes=142 > Combine input records=0 > Combine output records=0 > Reduce input groups=0 > Reduce shuffle bytes=88 > Reduce input records=0 > Reduce output records=0 > Spilled Records=1 > Shuffled Maps =2 > Failed Shuffles=0 > Merged Map outputs=2 > GC time elapsed (ms)=6 > CPU time spent (ms)=0 > Physical memory (bytes) snapshot=0 > Virtual memory (bytes) snapshot=0 > Total committed heap usage (bytes)=531628032 > Shuffle Errors > BAD_ID=0 > CONNECTION=0 > IO_ERROR=0 > WRONG_LENGTH=0 > WRONG_MAP=0 > WRONG_REDUCE=0 > File Input Format Counters > Bytes Read=154 > File Output Format Counters > Bytes Written=0 > 2017-08-22 20:58:28,684 ERROR crawl.Generator - Generator: > java.io.IOException: Job failed! > at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:836) > at org.apache.nutch.crawl.Generator.generate(Generator.java:589) > at org.apache.nutch.crawl.Generator.run(Generator.java:764) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.nutch.crawl.Generator.main(Generator.java:717) >
