Is your hadoop jar in the lib directory named
"hadoop-0.4.0-patched.jar!" with the exclamation point? If it is, that
may be causing the error. Also let me know if you can ping the namenode
from any of the data nodes.
Dennis Kubes
cybercouf wrote:
I tried both with "localhost" or "myhostname.domaine" in the slaves file, but
still the same problem.
I used the DEBUG log mode in order to find a clue.
hadoop-nutch-datanode.log
2007-05-02 17:27:57,575 DEBUG conf.Configuration - java.io.IOException:
config()
at org.apache.hadoop.conf.Configuration.<init>(Configuration.java:67)
at org.apache.hadoop.dfs.DataNode.main(DataNode.java:951)
2007-05-02 17:27:57,709 INFO conf.Configuration - parsing
jar:file:/home/nutch/search/lib/hadoop-0.4.0-patched.jar!/hadoop-default.xml
2007-05-02 17:27:57,773 INFO conf.Configuration - parsing
file:/home/nutch/search/conf/hadoop-site.xml
2007-05-02 17:27:57,814 INFO dfs.DataNode - Opened server at 50010
2007-05-02 17:27:57,838 INFO dfs.DataNode - Namenode not available yet,
Zzzzz...
2007-05-02 17:28:07,848 INFO dfs.DataNode - Namenode not available yet,
Zzzzz...
hadoop-nutch-namenode.log
2007-05-02 17:27:56,422 DEBUG conf.Configuration - java.io.IOException:
config()
at org.apache.hadoop.conf.Configuration.<init>(Configuration.java:67)
at org.apache.hadoop.dfs.NameNode.main(NameNode.java:475)
2007-05-02 17:27:56,553 INFO conf.Configuration - parsing
jar:file:/home/nutch/search/lib/hadoop-0.4.0-patched.jar!/hadoop-default.xml
2007-05-02 17:27:56,618 INFO conf.Configuration - parsing
file:/home/nutch/search/conf/hadoop-site.xml
2007-05-02 17:27:56,716 INFO util.Credential - Checking Resource aliases
2007-05-02 17:27:56,729 INFO http.HttpServer - Version Jetty/5.1.4
2007-05-02 17:27:56,730 WARN servlet.WebApplicationContext - Web
application not found
/home/nutch/search/file:/home/nutch/search/lib/hadoop-0.4.0-patched.jar!/webapps/dfs
2007-05-02 17:27:56,731 WARN servlet.WebApplicationContext - Configuration
error on
/home/nutch/search/file:/home/nutch/search/lib/hadoop-0.4.0-patched.jar!/webapps/dfs
java.io.FileNotFoundException:
/home/nutch/search/file:/home/nutch/search/lib/hadoop-0.4.0-patched.jar!/webapps/dfs
at
org.mortbay.jetty.servlet.WebApplicationContext.resolveWebApp(WebApplicationContext.java:266)
[...]
at org.apache.hadoop.dfs.NameNode.main(NameNode.java:491)
2007-05-02 17:27:56,732 INFO util.Container - Started
HttpContext[/logs,/logs]
2007-05-02 17:27:56,732 INFO util.Container - Started
HttpContext[/static,/static]
2007-05-02 17:27:56,736 INFO http.SocketListener - Started SocketListener
on 0.0.0.0:50070
hadoop-nutch-tasktracker.log
2007-05-02 17:27:59,783 DEBUG conf.Configuration - java.io.IOException:
config()
at org.apache.hadoop.conf.Configuration.<init>(Configuration.java:67)
at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:66)
at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:1129)
2007-05-02 17:27:59,934 INFO conf.Configuration - parsing
jar:file:/home/nutch/search/lib/hadoop-0.4.0-patched.jar!/hadoop-default.xml
2007-05-02 17:28:00,038 INFO conf.Configuration - parsing
jar:file:/home/nutch/search/lib/hadoop-0.4.0-patched.jar!/mapred-default.xml
2007-05-02 17:28:00,040 INFO conf.Configuration - parsing
file:/home/nutch/search/conf/hadoop-site.xml
2007-05-02 17:28:00,156 INFO util.Credential - Checking Resource aliases
2007-05-02 17:28:00,169 INFO http.HttpServer - Version Jetty/5.1.4
2007-05-02 17:28:00,170 WARN servlet.WebApplicationContext - Web
application not found
/home/nutch/search/file:/home/nutch/search/lib/hadoop-0.4.0-patched.jar!/webapps/task
2007-05-02 17:28:00,171 WARN servlet.WebApplicationContext - Configuration
error on
/home/nutch/search/file:/home/nutch/search/lib/hadoop-0.4.0-patched.jar!/webapps/task
java.io.FileNotFoundException:
/home/nutch/search/file:/home/nutch/search/lib/hadoop-0.4.0-patched.jar!/webapps/task
at
org.mortbay.jetty.servlet.WebApplicationContext.resolveWebApp(WebApplicationContext.java:266)
[...]
at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:1130)
2007-05-02 17:28:00,172 INFO util.Container - Started
HttpContext[/logs,/logs]
2007-05-02 17:28:00,173 INFO util.Container - Started
HttpContext[/static,/static]
2007-05-02 17:28:00,176 INFO http.SocketListener - Started SocketListener
on 0.0.0.0:50060
2007-05-02 17:28:00,176 WARN mapred.TaskTracker - Can not start task
tracker because Problem starting http server
hadoop-nutch-jobtracker.log
2007-05-02 17:27:58,631 DEBUG conf.Configuration - java.io.IOException:
config()
at org.apache.hadoop.conf.Configuration.<init>(Configuration.java:67)
at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:1132)
2007-05-02 17:27:58,785 INFO conf.Configuration - parsing
jar:file:/home/nutch/search/lib/hadoop-0.4.0-patched.jar!/hadoop-default.xml
2007-05-02 17:27:58,862 INFO conf.Configuration - parsing
file:/home/nutch/search/conf/hadoop-site.xml
2007-05-02 17:27:58,893 DEBUG conf.Configuration - java.io.IOException:
config(config)
at org.apache.hadoop.conf.Configuration.<init>(Configuration.java:76)
at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:86)
at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:427)
at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:65)
at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:1133)
2007-05-02 17:27:58,896 INFO conf.Configuration - parsing
jar:file:/home/nutch/search/lib/hadoop-0.4.0-patched.jar!/hadoop-default.xml
2007-05-02 17:27:58,907 INFO conf.Configuration - parsing
jar:file:/home/nutch/search/lib/hadoop-0.4.0-patched.jar!/mapred-default.xml
2007-05-02 17:27:58,922 INFO conf.Configuration - parsing
file:/home/nutch/search/conf/hadoop-site.xml
2007-05-02 17:27:58,989 WARN mapred.JobTracker - Starting tracker
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
[...]
at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:65)
at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:1133)
2007-05-02 17:27:58,990 WARN fs.DFSClient - Problem renewing lease for
DFSClient_515895079: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
[...]
at org.apache.hadoop.dfs.$Proxy0.renewLease(Unknown Source)
at org.apache.hadoop.dfs.DFSClient$LeaseChecker.run(DFSClient.java:437)
[...]
What errors are you seeing in your hadoop-namenode and datanode logs?
Dennis Kubes
