Hi,
After working on Hadoop for a while, I though I would integrate with
Eclipse and give that a shot. I am seeing a seemingly trivial issue..but,
could not figure out what is going on. I have tried googling and despite
those, I am unable to fix my issue. Any suggestion on the following would be
appreciated.
- Have a MAC with Hadoop 0.20.2-cdh3u0, java version 1.6.0_26, Eclipse
Indigo release.
- Hadoop normally runs fine - jps shows all the daemons running. Able to
see namenode and jobtracker on the web interface :
http://localhost:50070and 50030. (that makes me wonder if I have to
be using localhost or PDMac
as my hostname in Eclipse).
- Mapreduce on port 9001 and dfs on port 9000 as the xml configs.
- NOTE: my host name is PDMac which I had initially changed..using "sudo
scutil --set Hostname PDMac". I am not sure if this is an issue.
- I configured Eclipse Hadoop plugin "appropriately": I see a mapreduce -
elephant logo, create DFS locations - actually 2: one with localhost and
the other for PDMac.
- For Map/Reduce Master, I entered Host: localhost and port: 9001. For
DFS: localhost, 9000. Call to localhost/127.0.0.1:9000 failed on local
exception: java.io.EOFException.
- Then, I tried with my current running hostname. I entered Host: PDMac
and port: 9001. For DFS: PDMac, 9000. ==> Error: Call to PDMac/
192.168.1.102:9000 Failed on connection exception
Java.net.ConnectException. Connection refused.
1. I checked if this was something to do with /etc/hosts. I entered
192.168.1.102 as PDMac. I get same error at Eclipse.
2. I checked if this was due to ssh. I did "ssh localhost" and
immediately got a response "last login: Sep 11...."
3. But, "ssh PDMac" does not respond. Is that an issue? Because nodes in
dfs need ssh to connect...
4. I checked the namenode logs:
************************************************************/
2011-09-11 07:33:07,002 INFO
org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = PDMac/192.168.1.102
STARTUP_MSG: args = []
STARTUP_MSG: version = 0.20.2-cdh3u0
STARTUP_MSG: build = -r 81256ad0f2e4ab2bd34b04f53d25a6c23686dd14;
compiled by 'hudson' on Fri Mar 25 19:56:23 PDT 2011
************************************************************/
2011-09-11 07:33:07,851 INFO org.apache.hadoop.ipc.Server: Starting Socket
Reader #1 for port 9000
2011-09-11 07:33:07,854 INFO org.apache.hadoop.ipc.metrics.RpcMetrics:
Initializing RPC Metrics with hostName=NameNode, port=9000
2011-09-11 07:33:07,856 INFO
org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC Metrics
with hostName=NameNode, port=9000
*2011-09-11 07:33:07,866 INFO
org.apache.hadoop.hdfs.server.namenode.NameNode: Namenode up at: localhost/
127.0.0.1:9000*
2011-09-11 07:33:07,987 INFO org.mortbay.log: Logging to
org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
org.mortbay.log.Slf4jLog
.........
2011-09-11 07:33:22,822 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* ask
127.0.0.1:50010 to delete blk_8310136671599400924_1002
2011-09-11 07:34:05,136 WARN org.apache.hadoop.ipc.Server: Incorrect header
or version mismatch from 127.0.0.1:60733 got version 3 expected version 4
2011-09-11 07:34:08,614 WARN org.apache.hadoop.ipc.Server: Incorrect header
or version mismatch from 127.0.0.1:60734 got version 3 expected version 4
2011-09-11 07:34:15,228 WARN org.apache.hadoop.ipc.Server: Incorrect header
or version mismatch from 127.0.0.1:60735 got version 3 expected version 4
2011-09-11 07:38:12,349 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from
127.0.0.1
===========
So, I am not sure what is going on. First, I do not know what is my server =
localhost (127.0.0.1) or PDMac: 192.168.1.102 . Then, config options....I
think 9001 and 9000 are right ...as my hadoop/conf/ core-site.xml and dfs
xml says.
Any suggestions would be very much appreciated.
Thanks,
PD.
