Well, ERROR log is probably all we need to see, this is the application's main 
stdout which doesnt tell what is wrong.

M.

 
 
-----Original message-----
> From:shubham.gupta <shubham.gu...@orkash.com>
> Sent: Friday 14th October 2016 12:53
> To: user@nutch.apache.org
> Subject: Re: Injector and Generator Job Failing
> 
> Due to huge amount of hadoop logging I had only allowed the logging of 
> ERROR messages and above of both hadoop and nutch. Also, enabled the 
> periodic deletion of logs as lot of disk was being utilized. So, I am 
> kind of in the dark here.
> 
> Thanks and Regards,
> Shubham Gupta
> 
> On Friday 14 October 2016 04:11 PM, Markus Jelsma wrote:
> > Check the logs, this only tells you that i failed, not why.
> > M.
> >
> >   
> >   
> > -----Original message-----
> >> From:shubham.gupta <shubham.gu...@orkash.com>
> >> Sent: Friday 14th October 2016 12:15
> >> To: user@nutch.apache.org
> >> Subject: Injector and Generator Job Failing
> >>
> >> Hey
> >>
> >> Whenever i run the nutch application, only the injector and generate job
> >> fails.
> >> The path of the plugin folders in conf/nutch-site.xml is correct.
> >>
> >> The following error occurs:
> >> INFO mapreduce.Job: Job job_1476273924585_1272 failed with state FAILED
> >> due to: Task failed task_1476273924585_1272_m_000000
> >> Job failed as tasks failed. failedMaps:1 failedReduces:0
> >>
> >> INFO mapreduce.Job: Counters: 9
> >>       Job Counters
> >>           Failed map tasks=4
> >>           Launched map tasks=4
> >>           Other local map tasks=3
> >>           Data-local map tasks=1
> >>           Total time spent by all maps in occupied slots (ms)=17594276
> >>           Total time spent by all reduces in occupied slots (ms)=0
> >>           Total time spent by all map tasks (ms)=2513468
> >>           Total vcore-seconds taken by all map tasks=2513468
> >>           Total megabyte-seconds taken by all map tasks=8978107696
> >> ERROR crawl.*InjectorJob*: InjectorJob: java.lang.RuntimeException: job
> >> failed: name=inject jobid=job_1476273924585_1272
> >>       at 
> >> org.apache.nutch.util.NutchJob.waitForCompletion(NutchJob.java:120)
> >>       at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:231)
> >>       at org.apache.nutch.crawl.InjectorJob.inject(InjectorJob.java:252)
> >>       at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:275)
> >>       at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> >>       at org.apache.nutch.crawl.InjectorJob.main(InjectorJob.java:284)
> >>       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >>       at
> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >>       at
> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >>       at java.lang.reflect.Method.invoke(Method.java:606)
> >>       at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> >>       at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> >>
> >> mapreduce.Job: Job job_1476273924585_0973 failed with state FAILED due
> >> to: Task failed task_1476273924585_0973_m_000000
> >> Job failed as tasks failed. failedMaps:1 failedReduces:0
> >>
> >> INFO mapreduce.Job: Counters: 8
> >>           Job Counters
> >>                   Failed map tasks=4
> >>                   Launched map tasks=4
> >>                   Other local map tasks=4
> >>                   Total time spent by all maps in occupied slots
> >> (ms)=17585463
> >>                   Total time spent by all reduces in occupied slots (ms)=0
> >>                   Total time spent by all map tasks (ms)=2512209
> >>                   Total vcore-seconds taken by all map tasks=2512209
> >>                   Total megabyte-seconds taken by all map tasks=8973610548
> >> 16/10/14 04:15:49 ERROR crawl.*GeneratorJob*: GeneratorJob:
> >> java.lang.RuntimeException: job failed: name=generate:
> >> 1476396190-679771226, jobid=job_1476273924585_0973
> >>           at
> >> org.apache.nutch.util.NutchJob.waitForCompletion(NutchJob.java:120)
> >>           at org.apache.nutch.crawl.GeneratorJob.run(GeneratorJob.java:227)
> >>           at
> >> org.apache.nutch.crawl.GeneratorJob.generate(GeneratorJob.java:256)
> >>           at org.apache.nutch.crawl.GeneratorJob.run(GeneratorJob.java:322)
> >>           at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> >>           at 
> >> org.apache.nutch.crawl.GeneratorJob.main(GeneratorJob.java:330)
> >>           at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >>           at
> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >>           at
> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >>           at java.lang.reflect.Method.invoke(Method.java:606)
> >>           at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> >>           at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> >>
> >> Any suggestions.
> >>
> >> -- 
> >> Thanks and Regards,
> >> Shubham Gupta
> >>
> >>
> 
> 

Reply via email to