Hi Sir lewis
first of all thanks for your reply
when i run the inject step it works fine but when i do the generate step
i am getting the following errors
java.io.EOFException
at
org.apache.avro.io.BinaryDecoder.ensureBounds(BinaryDecoder.java:473)
at org.apache.avro.io.BinaryDecoder.readInt(BinaryDecoder.java:128)
at
org.apache.avro.io.BinaryDecoder.readBytes(BinaryDecoder.java:282)
at
org.apache.avro.io.ValidatingDecoder.readBytes(ValidatingDecoder.java:125)
at
org.apache.avro.generic.GenericDatumReader.readBytes(GenericDatumReader.java:368)
at
org.apache.avro.generic.GenericDatumReader.readBytes(GenericDatumReader.java:361)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:155)
at
org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:177)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:148)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:139)
at
org.apache.hadoop.io.serializer.avro.AvroSerialization$AvroDeserializer.deserialize(AvroSerialization.java:125)
at
org.apache.hadoop.mapreduce.task.ReduceContextImpl.nextKeyValue(ReduceContextImpl.java:145)
at
org.apache.hadoop.mapreduce.task.ReduceContextImpl.nextKey(ReduceContextImpl.java:121)
at
org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.nextKey(WrappedReducer.java:307)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:169)
at
org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:648)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:404)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:443)
2014-09-09 19:56:19,672 ERROR crawl.GeneratorJob - GeneratorJob:
java.lang.RuntimeException: job failed: name=generate: 23112,
jobid=job_local2049589210_0001
at
org.apache.nutch.util.NutchJob.waitForCompletion(NutchJob.java:54)
at org.apache.nutch.crawl.GeneratorJob.run(GeneratorJob.java:204)
at
org.apache.nutch.crawl.GeneratorJob.generate(GeneratorJob.java:228)
at org.apache.nutch.crawl.GeneratorJob.run(GeneratorJob.java:284)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.nutch.crawl.GeneratorJob.main(GeneratorJob.java:292)
On Tuesday 09 September 2014 09:15 PM, Lewis John Mcgibbney wrote:
Hi Sachin,
On Tue, Sep 9, 2014 at 8:38 AM, <[email protected]> wrote:
hi all i am trying to make nutch2.3 compatible with hadoop 2 so now i am
facing some problems.
I have configured apache gora0.4 and hbase 0.94 with nutch2.3
so now when i inject the urls in the database a table is created but when
i do the generate step my job is failing when i run my job in local mode
We're about to release Apache Gora 0.5 which will provide shim layers [0]
for backwards compatable support of multiple Hadoop versions.
It would really help ifg you could post what kind of errors you are getting
in local mode, we can then log them, track them and try to resolvce them
when Nutch 2.3-SNAPSHOT upgrades to Gora 0.5 prior to release of Nutch 2.3.
Thanks
Lewis
[0] https://issues.apache.org/jira/browse/GORA-346
