I tried all of the hadoop home dirs but didn't worke Mohammad Alkahtani P.O.Box 102275 Riyadh 11675 Saudi Arabia mobile: 00966 555 33 1717
On Sun, Mar 17, 2013 at 9:57 PM, Mohammad Alkahtani <[email protected]>wrote: > OK what the Hadoop home should be in ubuntu because the binary files in > /usr/bin > the hadoop-env.sh and othe xml file in /etc/hadoop > the conf files in /usr/share/hadoop/templates/conf > > shall I use /usr as hadoop path because it is the dir that contain the bin > files > > Mohammad Alkahtani > P.O.Box 102275 > Riyadh 11675 > Saudi Arabia > mobile: 00966 555 33 1717 > > > On Sun, Mar 17, 2013 at 9:50 PM, Mohammad Tariq <[email protected]>wrote: > >> log out from the user. log in again and see if it works. >> >> Warm Regards, >> Tariq >> https://mtariq.jux.com/ >> cloudfront.blogspot.com >> >> >> On Mon, Mar 18, 2013 at 12:18 AM, Mohammad Tariq <[email protected]>wrote: >> >>> you can avoid the warning by setting the following prop to true in the >>> hadoop-env.sh file : >>> export HADOOP_HOME_WARN_SUPPRESS=true >>> >>> >>> >>> Warm Regards, >>> Tariq >>> https://mtariq.jux.com/ >>> cloudfront.blogspot.com >>> >>> >>> On Mon, Mar 18, 2013 at 12:07 AM, Mohammad Alkahtani < >>> [email protected]> wrote: >>> >>>> Thank you Mohammad >>>> I still get the same error with this msg >>>> >>>> localhost: Warning: $HADOOP_HOME is deprecated. >>>> I searched ~/.bashrc but only what I wrote is there. >>>> >>>> >>>> Mohammad Alkahtani >>>> P.O.Box 102275 >>>> Riyadh 11675 >>>> Saudi Arabia >>>> mobile: 00966 555 33 1717 >>>> >>>> >>>> On Sun, Mar 17, 2013 at 9:21 PM, Mohammad Tariq <[email protected]>wrote: >>>> >>>>> you can do that using these command : >>>>> >>>>> sudo gedit ~/.bashrc >>>>> >>>>> then go to the end of the file and add this line : >>>>> export HADOOP_HOME=/YOUR_FULL_HADOOP_PATH >>>>> >>>>> after that use it to freeze the changes : >>>>> source ~/.bashrc >>>>> >>>>> to check it : >>>>> echo $HADOOP_HOME >>>>> >>>>> This will permanently set your HADOOP_HOME. >>>>> >>>>> HTH >>>>> >>>>> >>>>> Warm Regards, >>>>> Tariq >>>>> https://mtariq.jux.com/ >>>>> cloudfront.blogspot.com >>>>> >>>>> >>>>> On Sun, Mar 17, 2013 at 11:46 PM, Mohammad Alkahtani < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Tariq, Could you please tell me how to set HADOOP_HOME because I >>>>>> don't find it in the hadoop-env.sh >>>>>> >>>>>> Thank you Shashwat >>>>>> this is the output and it is already configured but hadoop don't >>>>>> read the configuration from here. >>>>>> >>>>>> /usr/share/maven-repo/org/apache >>>>>> /commons/commons-parent/22/commons-parent-22-site.xml >>>>>> /usr/share/maven-repo/org/apache/commons/commons-parent/debian >>>>>> /commons-parent-debian-site.xml >>>>>> /usr/share/maven-repo/org/apache/apache/10/apache-10-site.xml >>>>>> /usr/share/maven-repo/org/apache/apache/debian/apache-debian-site.xml >>>>>> /usr/share/compiz/composite.xml >>>>>> /usr/share/hadoop/templates/conf/mapred-site.xml >>>>>> /usr/share/hadoop/templates/conf/core-site.xml >>>>>> /usr/share/hadoop/templates/conf/hdfs-site.xml >>>>>> >>>>>> Mohammad Alkahtani >>>>>> P.O.Box 102275 >>>>>> Riyadh 11675 >>>>>> Saudi Arabia >>>>>> mobile: 00966 555 33 1717 >>>>>> >>>>>> >>>>>> On Sun, Mar 17, 2013 at 9:07 PM, shashwat shriparv < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> try >>>>>>> find / -type f -iname "*site.xml" >>>>>>> it will show you where ever those files are.. >>>>>>> >>>>>>> >>>>>>> >>>>>>> ∞ >>>>>>> Shashwat Shriparv >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Sun, Mar 17, 2013 at 11:34 PM, Mohammad Alkahtani < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> The problem is I tried I read the configuration file by changing >>>>>>>> export HADOOP_CONF_DIR=${HADOOP_CONF_ >>>>>>>> DIR:-"/usr/shar/hadoop/templates/conf"} >>>>>>>> but I think Hadoop dosen't get the configration from this dir, I >>>>>>>> trid and searched the system for conf dir the only dir is this one >>>>>>>> which I >>>>>>>> changed. >>>>>>>> >>>>>>>> Mohammad Alkahtani >>>>>>>> P.O.Box 102275 >>>>>>>> Riyadh 11675 >>>>>>>> Saudi Arabia >>>>>>>> mobile: 00966 555 33 1717 >>>>>>>> >>>>>>>> >>>>>>>> On Sun, Mar 17, 2013 at 8:57 PM, shashwat shriparv < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Ye its is asking for file:/// instead of hdfs:// just check if it >>>>>>>>> is taking setting configuration from other location... >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> ∞ >>>>>>>>> Shashwat Shriparv >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sun, Mar 17, 2013 at 11:07 PM, Luangsay Sourygna < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> What is the version of Hadoop you use? >>>>>>>>>> >>>>>>>>>> Try using fs.defaultFS instead of fs.default.name (see the list >>>>>>>>>> of all >>>>>>>>>> the deprecated properties here: >>>>>>>>>> >>>>>>>>>> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/DeprecatedProperties.html >>>>>>>>>> ). >>>>>>>>>> I remember I once had a similar error message and it was due to >>>>>>>>>> the >>>>>>>>>> change in properties names. >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> >>>>>>>>>> Sourygna >>>>>>>>>> >>>>>>>>>> On Sun, Mar 17, 2013 at 2:32 PM, Mohammad Alkahtani >>>>>>>>>> <[email protected]> wrote: >>>>>>>>>> > Hi to all users of Hadoop, >>>>>>>>>> > >>>>>>>>>> > I installed Hadoop the .deb file on Ubuntu 12.04 but I might >>>>>>>>>> could not >>>>>>>>>> > configure it right. The conf dir is under templates in >>>>>>>>>> /usr/shar/hadoop. I >>>>>>>>>> > edit the core-site.xml, mapred-site.xml files to give >>>>>>>>>> > <property> >>>>>>>>>> > <name>fs.default.name</name> >>>>>>>>>> > <value>hdfs://localhost:9000</value> >>>>>>>>>> > </property> >>>>>>>>>> > and for mapred >>>>>>>>>> > <property> >>>>>>>>>> > <name>mapred.job.tracker</name> >>>>>>>>>> > <value>localhost:9001</value> >>>>>>>>>> > </property> >>>>>>>>>> > >>>>>>>>>> > but i get these errors, I assume that there is problem, Hadoop >>>>>>>>>> cannot read >>>>>>>>>> > the configuration file. >>>>>>>>>> > I chaned the hadoop-env.sh to >>>>>>>>>> > export >>>>>>>>>> HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/usr/shar/hadoop/templates/conf"} >>>>>>>>>> > but dosen't solve the problem. >>>>>>>>>> > >>>>>>>>>> > ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: >>>>>>>>>> > java.lang.IllegalArgumentException: Does not contain a valid >>>>>>>>>> host:port >>>>>>>>>> > authority: file:/// at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164) at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:201) >>>>>>>>>> > at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:231) >>>>>>>>>> > at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getServiceAddress(NameNode.java:225) >>>>>>>>>> > at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:347) >>>>>>>>>> > at >>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.(DataNode.java:309) >>>>>>>>>> at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1651) >>>>>>>>>> > at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1590) >>>>>>>>>> > at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1608) >>>>>>>>>> > at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1734) >>>>>>>>>> > at >>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1751) >>>>>>>>>> > >>>>>>>>>> > ________________________________ >>>>>>>>>> > >>>>>>>>>> > FATAL org.apache.hadoop.mapred.JobTracker: >>>>>>>>>> > java.lang.IllegalArgumentException: Does not contain a valid >>>>>>>>>> host:port >>>>>>>>>> > authority: local at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164) at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:130) at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:2312) >>>>>>>>>> at >>>>>>>>>> > org.apache.hadoop.mapred.JobTracker.(JobTracker.java:2070) at >>>>>>>>>> > org.apache.hadoop.mapred.JobTracker.(JobTracker.java:1889) at >>>>>>>>>> > org.apache.hadoop.mapred.JobTracker.(JobTracker.java:1883) at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:312) >>>>>>>>>> at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:303) >>>>>>>>>> at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:298) >>>>>>>>>> > at >>>>>>>>>> org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4791) >>>>>>>>>> > >>>>>>>>>> > ________________________________ >>>>>>>>>> > >>>>>>>>>> > ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: >>>>>>>>>> > java.lang.IllegalArgumentException: Does not contain a valid >>>>>>>>>> host:port >>>>>>>>>> > authority: file:/// at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164) at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:201) >>>>>>>>>> > at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:231) >>>>>>>>>> > at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:265) >>>>>>>>>> > at >>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:536) >>>>>>>>>> at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1410) >>>>>>>>>> > at >>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1419) >>>>>>>>>> > >>>>>>>>>> > ________________________________ >>>>>>>>>> > >>>>>>>>>> > Exception in thread "main" java.lang.IllegalArgumentException: >>>>>>>>>> Does not >>>>>>>>>> > contain a valid host:port authority: file:/// at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164) at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:201) >>>>>>>>>> > at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:231) >>>>>>>>>> > at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getServiceAddress(NameNode.java:225) >>>>>>>>>> > at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.initialize(SecondaryNameNode.java:167) >>>>>>>>>> > at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.(SecondaryNameNode.java:135) >>>>>>>>>> > at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.main(SecondaryNameNode.java:650) >>>>>>>>>> > >>>>>>>>>> > ________________________________ >>>>>>>>>> > >>>>>>>>>> > ERROR org.apache.hadoop.mapred.TaskTracker: Can not start task >>>>>>>>>> tracker >>>>>>>>>> > because java.lang.IllegalArgumentException: Does not contain a >>>>>>>>>> valid >>>>>>>>>> > host:port authority: local at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164) at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:130) at >>>>>>>>>> > >>>>>>>>>> org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:2312) >>>>>>>>>> at >>>>>>>>>> > org.apache.hadoop.mapred.TaskTracker.(TaskTracker.java:1532) at >>>>>>>>>> > org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3906) >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > Regards, >>>>>>>>>> > Mohammad Alkahtani >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
