Hi,
I setup 3 machine Hadoop cluster using the instructions in:
https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html
When hadoop and yarn are running, if I run 'jps' on the three nodes, I see
on hadoop-master:
19696 SecondaryNameNode
20068 ResourceManager
20374 NodeManager
19383 DataNode
19257 NameNode
and on the two slaves:
17812 DataNode
18164 NodeManager
When I run inject using below command, it mostly succeeds, shows number of
urls injected.
apache-nutch-1.14/runtime/deploy/bin/nutch inject
/nutch/crawl/crawlDb /nutch/urls
When I run generate, it very often fails with a connection refused to an
unknown port. Sometimes, the 'select from crawlDb' job of generate
succeeds, and the 'partition' job fails. Sometimes, the 'select from
crawlDb' fails. All of them with the same connection refused to a random
port in localhost. The port number keeps changing with every attempt.
apache-nutch-1.14/runtime/deploy/bin/nutch generate
/nutch/crawl/crawlDb /nutch/crawl/segments
I have checked the configs multiple times, couldn't find the issue. Could
anyone please give me a pointer?
2018-02-27 11:09:51,955 INFO [main] mapreduce.Job
(Job.java:monitorAndPrintJob(1619)) - Running job: job_1519727841906_0003
2018-02-27 11:15:55,004 INFO [main] mapreduce.Job
(Job.java:monitorAndPrintJob(1640)) - Job job_1519727841906_0003 running in
uber mode : false
2018-02-27 11:15:55,006 INFO [main] mapreduce.Job
(Job.java:monitorAndPrintJob(1647)) - map 0% reduce 0%
2018-02-27 11:15:55,016 INFO [main] mapreduce.Job
(Job.java:monitorAndPrintJob(1660)) - Job job_1519727841906_0003 failed
with state FAILED due to: Application application_1519727841906_0003 failed
2 times due to Error launching appattempt_1519727841906_0003_000002. Got
exception: java.net.ConnectException: *Call From localhost/127.0.0.1
<http://127.0.0.1> to localhost:39093 failed on connection exception:
java.net.ConnectException: Connection refused*; For more details see:
http://wiki.apache.org/hadoop/ConnectionRefused
Thanks,
Sahas
