[
https://issues.apache.org/jira/browse/NUTCH-1084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15292406#comment-15292406
]
kaveh minooie commented on NUTCH-1084:
--------------------------------------
has there been any update on this issue? I am running master and this is what I
got :
crawler@d1r2n2:/2locos/nutch/deploy$ NUTCH_HOME=$( pwd ) bin/nutch readseg
-list /2locos/segments/20160511193155
WARNING: Use "yarn jar" to launch YARN applications.
16/05/19 16:55:42 INFO zlib.ZlibFactory: Successfully loaded & initialized
native-zlib library
16/05/19 16:55:42 INFO compress.CodecPool: Got brand-new decompressor [.deflate]
16/05/19 16:55:42 INFO compress.CodecPool: Got brand-new decompressor [.deflate]
16/05/19 16:55:42 INFO compress.CodecPool: Got brand-new decompressor [.deflate]
16/05/19 16:55:42 INFO compress.CodecPool: Got brand-new decompressor [.deflate]
16/05/19 16:55:42 INFO compress.CodecPool: Got brand-new decompressor [.deflate]
Exception in thread "main" java.io.IOException: can't find class:
org.apache.nutch.protocol.ProtocolStatus because
org.apache.nutch.protocol.ProtocolStatus
at
org.apache.hadoop.io.AbstractMapWritable.readFields(AbstractMapWritable.java:212)
at org.apache.hadoop.io.MapWritable.readFields(MapWritable.java:167)
at org.apache.nutch.crawl.CrawlDatum.readFields(CrawlDatum.java:317)
at
org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:2256)
at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:2384)
at org.apache.hadoop.io.MapFile$Reader.next(MapFile.java:673)
at
org.apache.nutch.segment.SegmentReader.getStats(SegmentReader.java:534)
at org.apache.nutch.segment.SegmentReader.list(SegmentReader.java:477)
at org.apache.nutch.segment.SegmentReader.main(SegmentReader.java:655)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
as you can see setting NUTCH_HOME has no effect. and if I set HADOOP_CLASSPATH
I get the same error that [~markus17] has posted 2 posts above, and [~ndouba]
branch is way behind the master. anybody has any idea what I should do ? (
short of converting [~ndouba] changes into a patch and try to apply it to
master)
> ReadDB url throws exception
> ---------------------------
>
> Key: NUTCH-1084
> URL: https://issues.apache.org/jira/browse/NUTCH-1084
> Project: Nutch
> Issue Type: Bug
> Affects Versions: 1.3
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Attachments: NUTCH-1084.patch
>
>
> Readdb -url suffers from two problems:
> 1. it trips over the _SUCCESS file generated by newer Hadoop version
> 2. throws can't find class: org.apache.nutch.protocol.ProtocolStatus (???)
> The first problem can be remedied by not allowing the injector or updater to
> write the _SUCCESS file. Until now that's the solution implemented for
> similar issues. I've not been successful as to make the Hadoop readers simply
> skip the file.
> The second issue seems a bit strange and did not happen on a local check out.
> I'm not yet sure whether this is a Hadoop issue or something being corrupt in
> the CrawlDB. Here's the stack trace:
> {code}
> Exception in thread "main" java.io.IOException: can't find class:
> org.apache.nutch.protocol.ProtocolStatus because
> org.apache.nutch.protocol.ProtocolStatus
> at
> org.apache.hadoop.io.AbstractMapWritable.readFields(AbstractMapWritable.java:204)
> at org.apache.hadoop.io.MapWritable.readFields(MapWritable.java:146)
> at org.apache.nutch.crawl.CrawlDatum.readFields(CrawlDatum.java:278)
> at
> org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:1751)
> at org.apache.hadoop.io.MapFile$Reader.get(MapFile.java:524)
> at
> org.apache.hadoop.mapred.MapFileOutputFormat.getEntry(MapFileOutputFormat.java:105)
> at org.apache.nutch.crawl.CrawlDbReader.get(CrawlDbReader.java:383)
> at
> org.apache.nutch.crawl.CrawlDbReader.readUrl(CrawlDbReader.java:389)
> at org.apache.nutch.crawl.CrawlDbReader.main(CrawlDbReader.java:514)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
