set these properties in the configuration files present in your /etc directory. and HADOOP_HOME is the parent directory of the hadoop bin directory that holds the Hadoop scripts. so, set that accordingly in .bashrc file.
Warm Regards, Tariq https://mtariq.jux.com/ cloudfront.blogspot.com On Mon, Mar 18, 2013 at 12:35 AM, Mohammad Alkahtani <[email protected]>wrote: > Thank you Mohammad Tariq > > Mohammad Alkahtani > P.O.Box 102275 > Riyadh 11675 > Saudi Arabia > mobile: 00966 555 33 1717 > > > On Sun, Mar 17, 2013 at 10:04 PM, Mohammad Alkahtani < > [email protected]> wrote: > >> I tried all of the hadoop home dirs but didn't worke >> >> Mohammad Alkahtani >> P.O.Box 102275 >> Riyadh 11675 >> Saudi Arabia >> mobile: 00966 555 33 1717 >> >> >> On Sun, Mar 17, 2013 at 9:57 PM, Mohammad Alkahtani < >> [email protected]> wrote: >> >>> OK what the Hadoop home should be in ubuntu because the binary files in >>> /usr/bin >>> the hadoop-env.sh and othe xml file in /etc/hadoop >>> the conf files in /usr/share/hadoop/templates/conf >>> >>> shall I use /usr as hadoop path because it is the dir that contain the >>> bin files >>> >>> Mohammad Alkahtani >>> P.O.Box 102275 >>> Riyadh 11675 >>> Saudi Arabia >>> mobile: 00966 555 33 1717 >>> >>> >>> On Sun, Mar 17, 2013 at 9:50 PM, Mohammad Tariq <[email protected]>wrote: >>> >>>> log out from the user. log in again and see if it works. >>>> >>>> Warm Regards, >>>> Tariq >>>> https://mtariq.jux.com/ >>>> cloudfront.blogspot.com >>>> >>>> >>>> On Mon, Mar 18, 2013 at 12:18 AM, Mohammad Tariq <[email protected]>wrote: >>>> >>>>> you can avoid the warning by setting the following prop to true in the >>>>> hadoop-env.sh file : >>>>> export HADOOP_HOME_WARN_SUPPRESS=true >>>>> >>>>> >>>>> >>>>> Warm Regards, >>>>> Tariq >>>>> https://mtariq.jux.com/ >>>>> cloudfront.blogspot.com >>>>> >>>>> >>>>> On Mon, Mar 18, 2013 at 12:07 AM, Mohammad Alkahtani < >>>>> [email protected]> wrote: >>>>> >>>>>> Thank you Mohammad >>>>>> I still get the same error with this msg >>>>>> >>>>>> localhost: Warning: $HADOOP_HOME is deprecated. >>>>>> I searched ~/.bashrc but only what I wrote is there. >>>>>> >>>>>> >>>>>> Mohammad Alkahtani >>>>>> P.O.Box 102275 >>>>>> Riyadh 11675 >>>>>> Saudi Arabia >>>>>> mobile: 00966 555 33 1717 >>>>>> >>>>>> >>>>>> On Sun, Mar 17, 2013 at 9:21 PM, Mohammad Tariq >>>>>> <[email protected]>wrote: >>>>>> >>>>>>> you can do that using these command : >>>>>>> >>>>>>> sudo gedit ~/.bashrc >>>>>>> >>>>>>> then go to the end of the file and add this line : >>>>>>> export HADOOP_HOME=/YOUR_FULL_HADOOP_PATH >>>>>>> >>>>>>> after that use it to freeze the changes : >>>>>>> source ~/.bashrc >>>>>>> >>>>>>> to check it : >>>>>>> echo $HADOOP_HOME >>>>>>> >>>>>>> This will permanently set your HADOOP_HOME. >>>>>>> >>>>>>> HTH >>>>>>> >>>>>>> >>>>>>> Warm Regards, >>>>>>> Tariq >>>>>>> https://mtariq.jux.com/ >>>>>>> cloudfront.blogspot.com >>>>>>> >>>>>>> >>>>>>> On Sun, Mar 17, 2013 at 11:46 PM, Mohammad Alkahtani < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi Tariq, Could you please tell me how to set HADOOP_HOME because >>>>>>>> I don't find it in the hadoop-env.sh >>>>>>>> >>>>>>>> Thank you Shashwat >>>>>>>> this is the output and it is already configured but hadoop don't >>>>>>>> read the configuration from here. >>>>>>>> >>>>>>>> /usr/share/maven-repo/org/apache >>>>>>>> /commons/commons-parent/22/commons-parent-22-site.xml >>>>>>>> /usr/share/maven-repo/org/apache/commons/commons-parent/debian >>>>>>>> /commons-parent-debian-site.xml >>>>>>>> /usr/share/maven-repo/org/apache/apache/10/apache-10-site.xml >>>>>>>> /usr/share/maven-repo/org/apache/apache/debian/apache-debian-site. >>>>>>>> xml >>>>>>>> /usr/share/compiz/composite.xml >>>>>>>> /usr/share/hadoop/templates/conf/mapred-site.xml >>>>>>>> /usr/share/hadoop/templates/conf/core-site.xml >>>>>>>> /usr/share/hadoop/templates/conf/hdfs-site.xml >>>>>>>> >>>>>>>> Mohammad Alkahtani >>>>>>>> P.O.Box 102275 >>>>>>>> Riyadh 11675 >>>>>>>> Saudi Arabia >>>>>>>> mobile: 00966 555 33 1717 >>>>>>>> >>>>>>>> >>>>>>>> On Sun, Mar 17, 2013 at 9:07 PM, shashwat shriparv < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> try >>>>>>>>> find / -type f -iname "*site.xml" >>>>>>>>> it will show you where ever those files are.. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> ∞ >>>>>>>>> Shashwat Shriparv >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sun, Mar 17, 2013 at 11:34 PM, Mohammad Alkahtani < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> The problem is I tried I read the configuration file by changing >>>>>>>>>> export HADOOP_CONF_DIR=${HADOOP_CONF_ >>>>>>>>>> DIR:-"/usr/shar/hadoop/templates/conf"} >>>>>>>>>> but I think Hadoop dosen't get the configration from this dir, I >>>>>>>>>> trid and searched the system for conf dir the only dir is this one >>>>>>>>>> which I >>>>>>>>>> changed. >>>>>>>>>> >>>>>>>>>> Mohammad Alkahtani >>>>>>>>>> P.O.Box 102275 >>>>>>>>>> Riyadh 11675 >>>>>>>>>> Saudi Arabia >>>>>>>>>> mobile: 00966 555 33 1717 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Sun, Mar 17, 2013 at 8:57 PM, shashwat shriparv < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Ye its is asking for file:/// instead of hdfs:// just check if >>>>>>>>>>> it is taking setting configuration from other location... >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> ∞ >>>>>>>>>>> Shashwat Shriparv >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Sun, Mar 17, 2013 at 11:07 PM, Luangsay Sourygna < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> What is the version of Hadoop you use? >>>>>>>>>>>> >>>>>>>>>>>> Try using fs.defaultFS instead of fs.default.name (see the >>>>>>>>>>>> list of all >>>>>>>>>>>> the deprecated properties here: >>>>>>>>>>>> >>>>>>>>>>>> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/DeprecatedProperties.html >>>>>>>>>>>> ). >>>>>>>>>>>> I remember I once had a similar error message and it was due to >>>>>>>>>>>> the >>>>>>>>>>>> change in properties names. >>>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> >>>>>>>>>>>> Sourygna >>>>>>>>>>>> >>>>>>>>>>>> On Sun, Mar 17, 2013 at 2:32 PM, Mohammad Alkahtani >>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>> > Hi to all users of Hadoop, >>>>>>>>>>>> > >>>>>>>>>>>> > I installed Hadoop the .deb file on Ubuntu 12.04 but I might >>>>>>>>>>>> could not >>>>>>>>>>>> > configure it right. The conf dir is under templates in >>>>>>>>>>>> /usr/shar/hadoop. I >>>>>>>>>>>> > edit the core-site.xml, mapred-site.xml files to give >>>>>>>>>>>> > <property> >>>>>>>>>>>> > <name>fs.default.name</name> >>>>>>>>>>>> > <value>hdfs://localhost:9000</value> >>>>>>>>>>>> > </property> >>>>>>>>>>>> > and for mapred >>>>>>>>>>>> > <property> >>>>>>>>>>>> > <name>mapred.job.tracker</name> >>>>>>>>>>>> > <value>localhost:9001</value> >>>>>>>>>>>> > </property> >>>>>>>>>>>> > >>>>>>>>>>>> > but i get these errors, I assume that there is problem, >>>>>>>>>>>> Hadoop cannot read >>>>>>>>>>>> > the configuration file. >>>>>>>>>>>> > I chaned the hadoop-env.sh to >>>>>>>>>>>> > export >>>>>>>>>>>> HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/usr/shar/hadoop/templates/conf"} >>>>>>>>>>>> > but dosen't solve the problem. >>>>>>>>>>>> > >>>>>>>>>>>> > ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: >>>>>>>>>>>> > java.lang.IllegalArgumentException: Does not contain a valid >>>>>>>>>>>> host:port >>>>>>>>>>>> > authority: file:/// at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164) >>>>>>>>>>>> at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:201) >>>>>>>>>>>> > at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:231) >>>>>>>>>>>> > at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getServiceAddress(NameNode.java:225) >>>>>>>>>>>> > at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:347) >>>>>>>>>>>> > at >>>>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.(DataNode.java:309) >>>>>>>>>>>> at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1651) >>>>>>>>>>>> > at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1590) >>>>>>>>>>>> > at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1608) >>>>>>>>>>>> > at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1734) >>>>>>>>>>>> > at >>>>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1751) >>>>>>>>>>>> > >>>>>>>>>>>> > ________________________________ >>>>>>>>>>>> > >>>>>>>>>>>> > FATAL org.apache.hadoop.mapred.JobTracker: >>>>>>>>>>>> > java.lang.IllegalArgumentException: Does not contain a valid >>>>>>>>>>>> host:port >>>>>>>>>>>> > authority: local at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164) >>>>>>>>>>>> at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:130) >>>>>>>>>>>> at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:2312) >>>>>>>>>>>> at >>>>>>>>>>>> > org.apache.hadoop.mapred.JobTracker.(JobTracker.java:2070) at >>>>>>>>>>>> > org.apache.hadoop.mapred.JobTracker.(JobTracker.java:1889) at >>>>>>>>>>>> > org.apache.hadoop.mapred.JobTracker.(JobTracker.java:1883) at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:312) >>>>>>>>>>>> at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:303) >>>>>>>>>>>> at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:298) >>>>>>>>>>>> > at >>>>>>>>>>>> org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4791) >>>>>>>>>>>> > >>>>>>>>>>>> > ________________________________ >>>>>>>>>>>> > >>>>>>>>>>>> > ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: >>>>>>>>>>>> > java.lang.IllegalArgumentException: Does not contain a valid >>>>>>>>>>>> host:port >>>>>>>>>>>> > authority: file:/// at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164) >>>>>>>>>>>> at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:201) >>>>>>>>>>>> > at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:231) >>>>>>>>>>>> > at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:265) >>>>>>>>>>>> > at >>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:536) >>>>>>>>>>>> at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1410) >>>>>>>>>>>> > at >>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1419) >>>>>>>>>>>> > >>>>>>>>>>>> > ________________________________ >>>>>>>>>>>> > >>>>>>>>>>>> > Exception in thread "main" >>>>>>>>>>>> java.lang.IllegalArgumentException: Does not >>>>>>>>>>>> > contain a valid host:port authority: file:/// at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164) >>>>>>>>>>>> at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:201) >>>>>>>>>>>> > at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:231) >>>>>>>>>>>> > at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getServiceAddress(NameNode.java:225) >>>>>>>>>>>> > at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.initialize(SecondaryNameNode.java:167) >>>>>>>>>>>> > at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.(SecondaryNameNode.java:135) >>>>>>>>>>>> > at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.main(SecondaryNameNode.java:650) >>>>>>>>>>>> > >>>>>>>>>>>> > ________________________________ >>>>>>>>>>>> > >>>>>>>>>>>> > ERROR org.apache.hadoop.mapred.TaskTracker: Can not start >>>>>>>>>>>> task tracker >>>>>>>>>>>> > because java.lang.IllegalArgumentException: Does not contain >>>>>>>>>>>> a valid >>>>>>>>>>>> > host:port authority: local at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164) >>>>>>>>>>>> at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:130) >>>>>>>>>>>> at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:2312) >>>>>>>>>>>> at >>>>>>>>>>>> > org.apache.hadoop.mapred.TaskTracker.(TaskTracker.java:1532) >>>>>>>>>>>> at >>>>>>>>>>>> > >>>>>>>>>>>> org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3906) >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > Regards, >>>>>>>>>>>> > Mohammad Alkahtani >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
