Hi, Sam: Thanks for your help! You are right, seems only one node is running. Here is the output:
xisisu@slave-01:/usr/local/hadoop$ hadoop dfsadmin -report DEPRECATED: Use of this script to execute hdfs command is deprecated. Instead use the hdfs command for it. OpenJDK 64-Bit Server VM warning: You have loaded library /usr/local/hadoop/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now. It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'. 14/07/13 12:50:14 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Configured Capacity: 14068822016 (13.10 GB) Present Capacity: 8425232425 (7.85 GB) DFS Remaining: 8423534592 (7.85 GB) DFS Used: 1697833 (1.62 MB) DFS Used%: 0.02% Under replicated blocks: 21 Blocks with corrupt replicas: 0 Missing blocks: 0 ------------------------------------------------- Datanodes available: 1 (1 total, 0 dead) Live datanodes: Name: 172.16.20.170:50010 (master) Hostname: master Decommission Status : Normal Configured Capacity: 14068822016 (13.10 GB) DFS Used: 1697833 (1.62 MB) Non DFS Used: 5643589591 (5.26 GB) DFS Remaining: 8423534592 (7.85 GB) DFS Used%: 0.01% DFS Remaining%: 59.87% Last contact: Sun Jul 13 12:50:13 CDT 2014 I get the same output when I run the command on the slave node. Is there anything I am missing in the config file? Thanks very much! Sisu On Sun, Jul 13, 2014 at 1:36 AM, Kilaru, Sambaiah < [email protected]> wrote: > HI Sisu Xi, > > On the master node can you check > Hadoop dfsadmin –report > And listing all the slave nodes or you can check master URL and it should > all datanodes listed as slave nodes. > Check for RM UI and slave nodes listed there also. > > Thanks, > Sam > > From: Sisu Xi <[email protected]> > Reply-To: "[email protected]" <[email protected]> > Date: Sunday, July 13, 2014 at 11:28 AM > To: "[email protected]" <[email protected]> > Subject: hadoop multinode, only master node doing the work > > Hi, all: > > I am new to hadoop. I followed the tutorial on > > http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/ > > and installed hadoop 2.2.0 on two 4-core Ubuntu 12.04 machines. > > I can start the pi program, however, only the master node is doing the > work (I checked the top on each machine). > Seems the two nodes are configured correctly, because I can start the > program in the slave node as well, and still only the master node is doing > the actual work. > I have tried different number of mappers for the pi program, and the > results is the same. > > Is there anything else I can check? > > In the end is my configure file on each host. > > Thanks very much! > > Sisu > > ---------yarn-site.xml------- > > <property> > <name>yarn.nodemanager.aux-services</name> > <value>mapreduce_shuffle</value> > </property> > > > <property> > <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name> > <value>org.apache.hadoop.mapred.ShuffleHandler</value> > </property> > > > <property> > <name>yarn.resourcemanager.address</name> > <value>master:8032</value> > </property> > > ---------------hdfs-site.xml-------------------- > > <property> > <name>dfs.replication</name> > <value>2</value> > </property> > > > <property> > <name>dfs.namenode.name.dir</name> > <value>file:/home/xisisu/mydata/hdfs/namenode</value> > </property> > > > <property> > <name>dfs.datanode.data.dir</name> > <value>file:/home/xisisu/mydata/hdfs/datanode</value> > </property> > > > -------------core-site.xml------------- > > <property> > <name>fs.default.name</name> > <value>hdfs://master:9000</value> > </property> > > ------------------mapred-site.xml----------------- > > <property> > <name>mapreduce.framework.name</name> > <value>yarn</value> > </property> > > <property> > <name>mapred.job.tracker</name> > <value>master:54311</value> > <description>The host and port that the MapReduce job tracker runs > at. If "local", then jobs are run in-process as a single map > and reduce task. > </description> > </property> > > > > > -- > > > *Sisu Xi, PhD Candidate* > > http://www.cse.wustl.edu/~xis/ > Department of Computer Science and Engineering > Campus Box 1045 > Washington University in St. Louis > One Brookings Drive > St. Louis, MO 63130 > -- *Sisu Xi, PhD Candidate* http://www.cse.wustl.edu/~xis/ Department of Computer Science and Engineering Campus Box 1045 Washington University in St. Louis One Brookings Drive St. Louis, MO 63130
