Hi, I've a simple hadoop setup with a name node and 1 data node. Using nutch 1.8 I'm able to walk through all steps: inject, generate, fetch, parse, updatedb, fetch-again, parse-again, ..., invertlinks.
But if I try to run the indexer using:
% ./bin/nutch index crawl/db/ -linkdb crawl/linkdb/ -dir crawl/segments
then I get an exception from hadoop:
Error: java.lang.IllegalArgumentException: n must be positive
at java.util.Random.nextInt(Random.java:300)
at org.elasticsearch.common.Names.randomNodeName(Names.java:45)
at
org.elasticsearch.node.internal.InternalSettingsPreparer.prepareSettings(InternalSettingsPreparer.java:119)
at
org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:159)
at
org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:125)
at
org.apache.nutch.indexwriter.elastic.ElasticIndexWriter.open(ElasticIndexWriter.java:106)
at org.apache.nutch.indexer.IndexWriters.open(IndexWriters.java:78)
at
org.apache.nutch.indexer.IndexerOutputFormat.getRecordWriter(IndexerOutputFormat.java:39)
at
org.apache.hadoop.mapred.ReduceTask$OldTrackingRecordWriter.<init>(ReduceTask.java:502)
at
org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:432)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)
I've attached the whole stack trace to the mail.
Searching the internet the error suggests that hadoop is unable to find
the data node but if that is the case shouldn't the error occur right
from the start?
Any ideas?
Regards,
Jens
--
28. Wonnemond 2014, 11:56
Homepage : http://www.wegtam.com
Real Programmers think better when playing Adventure or Rogue.
nutch-hadoop-elastic-stacktrace.log
Description: Binary data
pgpOyZse0AaKJ.pgp
Description: PGP signature
