keep the configuration same in the datanodes as well for the time being..Only thing that data node or slave machine should know is Masters files ( that means who is the master) and you need to tell the slave machine where is your namenode running, which you need to specify in the property fs.default.name and also you need to tell where is your job tracker running which you need to specify by the property mapred.job.tracker
Hope you might be able to bring up your cluster now.. if you still face the issues, you can follow my blog http://ksssblogs.blogspot.in/2013/07/multi-node-hadoop-cluster-set-using-vms.html Regards, Som Shekhar Sharma +91-8197243810 On Thu, Aug 8, 2013 at 5:21 PM, Shekhar Sharma <[email protected]>wrote: > if you have removed this property from the slave machines then your DN > information will be created under /tmp folder and once you reboot your data > node machines, the information will be lost.. > > Sorry i have not seen the logs..but you dont have play around the > properties.. > ...see datanode will not come up in scenario, where it is not able to send > the heart beat signal to the name node at port 54310 > > > > > Do step by step : > > Check whether you can ping every machine and you can do SSH in password > less manner > > Lets say i have one master machine whose hostname is *Master* and i have > two slave machines *Slave0* and *Slave1* ( i am assuming the OS used are > CentOS) > > In *master Machine* do the following things: > > *First disable the firewall by running the following command: > *as a root user run the following commands > service iptables save > service iptables stop > chkconfig iptables off > > specify the following properties in the corresponding files > > *mapred-site.xml* > > - mapred.job.tracker (Master:54311) > > *core-site.xml* > > - fs.default.name (hdfs://Master:54310) > - hadoop.tmp.dir (choose some persistent directory) > > *hdfs-site.xml* > > - dfs.replication (3) > - dfs.block.size(64MB) > > *Masters file* > > - Master > > *Slaves file* > > - Slave0 > - Slave1 > > *hadoop-env.sh* > > - export JAVA_HOME=<Your java home directory> > > > In *slave0 machine > * > > - Disable the firewall > - Same properties as you did in Masters machine > > In *slave 1 machine* > > - Disable the firewall > - Same properties as you did in Master machine > > > Once you start the cluster by running the command start-all.sh, check the > ports 54310 and 54311 got opened by running the command "netstat > -tuplen"..it will show whether ports are opened or not > > Regards, > Som Shekhar Sharma > +91-8197243810 > > > On Thu, Aug 8, 2013 at 4:57 PM, Felipe Gutierrez < > [email protected]> wrote: > >> Thanks, >> at all files I changed to master (cloud6) and I take off this property >> <name>hadoop.tmp.dir</name>. >> >> Felipe >> >> >> On Wed, Aug 7, 2013 at 3:20 PM, Shekhar Sharma <[email protected]>wrote: >> >>> Disable the firewall on data node and namenode machines.. >>> Regards, >>> Som Shekhar Sharma >>> +91-8197243810 >>> >>> >>> On Wed, Aug 7, 2013 at 11:33 PM, Jitendra Yadav < >>> [email protected]> wrote: >>> >>>> Your hdfs name entry should be same on master and databnodes >>>> >>>> * <name>fs.default.name</name>* >>>> *<value>hdfs://cloud6:54310</value>* >>>> >>>> Thanks >>>> On Wed, Aug 7, 2013 at 11:05 PM, Felipe Gutierrez < >>>> [email protected]> wrote: >>>> >>>>> on my slave the process is running: >>>>> hduser@cloud15:/usr/local/hadoop$ jps >>>>> 19025 DataNode >>>>> 19092 Jps >>>>> >>>>> >>>>> On Wed, Aug 7, 2013 at 2:26 PM, Jitendra Yadav < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Your logs showing that the process is creating IPC call not for >>>>>> namenode, it is hitting datanode itself. >>>>>> >>>>>> Check you please check you datanode processes status?. >>>>>> >>>>>> Regards >>>>>> Jitendra >>>>>> >>>>>> On Wed, Aug 7, 2013 at 10:29 PM, Felipe Gutierrez < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi everyone, >>>>>>> >>>>>>> My slave machine (cloud15) the datanode shows this log. It doesn't >>>>>>> connect to the master (cloud6). >>>>>>> >>>>>>> 2013-08-07 13:44:03,110 INFO org.apache.hadoop.ipc.Client: >>>>>>> Retrying connect to server: cloud15/192.168.188.15:54310. Already >>>>>>> tried 9 time(s); retry policy is >>>>>>> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS) >>>>>>> 2013-08-07 13:44:03,110 INFO org.apache.hadoop.ipc.RPC: Server at >>>>>>> cloud15/192.168.188.15:54310 not available yet, Zzzzz... >>>>>>> >>>>>>> But when I type jps command on slave machine DataNode is running. >>>>>>> This is my file core-site.xml in slave machine (cloud15): >>>>>>> <configuration> >>>>>>> <property> >>>>>>> <name>hadoop.tmp.dir</name> >>>>>>> <value>/app/hadoop/tmp</value> >>>>>>> <description>A base for other temporary directories.</description> >>>>>>> </property> >>>>>>> >>>>>>> <property> >>>>>>> <name>fs.default.name</name> >>>>>>> <value>hdfs://cloud15:54310</value> >>>>>>> <description>The name of the default file system. A URI whose >>>>>>> scheme and authority determine the FileSystem implementation. The >>>>>>> uri's scheme determines the config property (fs.SCHEME.impl) naming >>>>>>> the FileSystem implementation class. The uri's authority is used >>>>>>> to >>>>>>> determine the host, port, etc. for a filesystem.</description> >>>>>>> </property> >>>>>>> </configuration> >>>>>>> >>>>>>> In the master machine I just swap cloud15 to cloud6. >>>>>>> In the file /etc/host I have (192.168.188.15 cloud15) and >>>>>>> (192.168.188.6 cloud6) lines, and both machines access through ssh >>>>>>> with >>>>>>> out password. >>>>>>> >>>>>>> Am I missing anything? >>>>>>> >>>>>>> Thanks in advance! >>>>>>> Felipe >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *-- >>>>>>> -- Felipe Oliveira Gutierrez >>>>>>> -- [email protected] >>>>>>> -- https://sites.google.com/site/lipe82/Home/diaadia* >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> *-- >>>>> -- Felipe Oliveira Gutierrez >>>>> -- [email protected] >>>>> -- https://sites.google.com/site/lipe82/Home/diaadia* >>>>> >>>> >>>> >>> >> >> >> -- >> *-- >> -- Felipe Oliveira Gutierrez >> -- [email protected] >> -- https://sites.google.com/site/lipe82/Home/diaadia* >> > >
