Thank you Mohammad Tariq Mohammad Alkahtani P.O.Box 102275 Riyadh 11675 Saudi Arabia mobile: 00966 555 33 1717
On Sun, Mar 17, 2013 at 10:04 PM, Mohammad Alkahtani <[email protected]>wrote: > I tried all of the hadoop home dirs but didn't worke > > Mohammad Alkahtani > P.O.Box 102275 > Riyadh 11675 > Saudi Arabia > mobile: 00966 555 33 1717 > > > On Sun, Mar 17, 2013 at 9:57 PM, Mohammad Alkahtani <[email protected] > > wrote: > >> OK what the Hadoop home should be in ubuntu because the binary files in >> /usr/bin >> the hadoop-env.sh and othe xml file in /etc/hadoop >> the conf files in /usr/share/hadoop/templates/conf >> >> shall I use /usr as hadoop path because it is the dir that contain the >> bin files >> >> Mohammad Alkahtani >> P.O.Box 102275 >> Riyadh 11675 >> Saudi Arabia >> mobile: 00966 555 33 1717 >> >> >> On Sun, Mar 17, 2013 at 9:50 PM, Mohammad Tariq <[email protected]>wrote: >> >>> log out from the user. log in again and see if it works. >>> >>> Warm Regards, >>> Tariq >>> https://mtariq.jux.com/ >>> cloudfront.blogspot.com >>> >>> >>> On Mon, Mar 18, 2013 at 12:18 AM, Mohammad Tariq <[email protected]>wrote: >>> >>>> you can avoid the warning by setting the following prop to true in the >>>> hadoop-env.sh file : >>>> export HADOOP_HOME_WARN_SUPPRESS=true >>>> >>>> >>>> >>>> Warm Regards, >>>> Tariq >>>> https://mtariq.jux.com/ >>>> cloudfront.blogspot.com >>>> >>>> >>>> On Mon, Mar 18, 2013 at 12:07 AM, Mohammad Alkahtani < >>>> [email protected]> wrote: >>>> >>>>> Thank you Mohammad >>>>> I still get the same error with this msg >>>>> >>>>> localhost: Warning: $HADOOP_HOME is deprecated. >>>>> I searched ~/.bashrc but only what I wrote is there. >>>>> >>>>> >>>>> Mohammad Alkahtani >>>>> P.O.Box 102275 >>>>> Riyadh 11675 >>>>> Saudi Arabia >>>>> mobile: 00966 555 33 1717 >>>>> >>>>> >>>>> On Sun, Mar 17, 2013 at 9:21 PM, Mohammad Tariq <[email protected]>wrote: >>>>> >>>>>> you can do that using these command : >>>>>> >>>>>> sudo gedit ~/.bashrc >>>>>> >>>>>> then go to the end of the file and add this line : >>>>>> export HADOOP_HOME=/YOUR_FULL_HADOOP_PATH >>>>>> >>>>>> after that use it to freeze the changes : >>>>>> source ~/.bashrc >>>>>> >>>>>> to check it : >>>>>> echo $HADOOP_HOME >>>>>> >>>>>> This will permanently set your HADOOP_HOME. >>>>>> >>>>>> HTH >>>>>> >>>>>> >>>>>> Warm Regards, >>>>>> Tariq >>>>>> https://mtariq.jux.com/ >>>>>> cloudfront.blogspot.com >>>>>> >>>>>> >>>>>> On Sun, Mar 17, 2013 at 11:46 PM, Mohammad Alkahtani < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Tariq, Could you please tell me how to set HADOOP_HOME because I >>>>>>> don't find it in the hadoop-env.sh >>>>>>> >>>>>>> Thank you Shashwat >>>>>>> this is the output and it is already configured but hadoop don't >>>>>>> read the configuration from here. >>>>>>> >>>>>>> /usr/share/maven-repo/org/apache >>>>>>> /commons/commons-parent/22/commons-parent-22-site.xml >>>>>>> /usr/share/maven-repo/org/apache/commons/commons-parent/debian >>>>>>> /commons-parent-debian-site.xml >>>>>>> /usr/share/maven-repo/org/apache/apache/10/apache-10-site.xml >>>>>>> /usr/share/maven-repo/org/apache/apache/debian/apache-debian-site. >>>>>>> xml >>>>>>> /usr/share/compiz/composite.xml >>>>>>> /usr/share/hadoop/templates/conf/mapred-site.xml >>>>>>> /usr/share/hadoop/templates/conf/core-site.xml >>>>>>> /usr/share/hadoop/templates/conf/hdfs-site.xml >>>>>>> >>>>>>> Mohammad Alkahtani >>>>>>> P.O.Box 102275 >>>>>>> Riyadh 11675 >>>>>>> Saudi Arabia >>>>>>> mobile: 00966 555 33 1717 >>>>>>> >>>>>>> >>>>>>> On Sun, Mar 17, 2013 at 9:07 PM, shashwat shriparv < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> try >>>>>>>> find / -type f -iname "*site.xml" >>>>>>>> it will show you where ever those files are.. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ∞ >>>>>>>> Shashwat Shriparv >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Sun, Mar 17, 2013 at 11:34 PM, Mohammad Alkahtani < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> The problem is I tried I read the configuration file by changing >>>>>>>>> export HADOOP_CONF_DIR=${HADOOP_CONF_ >>>>>>>>> DIR:-"/usr/shar/hadoop/templates/conf"} >>>>>>>>> but I think Hadoop dosen't get the configration from this dir, I >>>>>>>>> trid and searched the system for conf dir the only dir is this one >>>>>>>>> which I >>>>>>>>> changed. >>>>>>>>> >>>>>>>>> Mohammad Alkahtani >>>>>>>>> P.O.Box 102275 >>>>>>>>> Riyadh 11675 >>>>>>>>> Saudi Arabia >>>>>>>>> mobile: 00966 555 33 1717 >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sun, Mar 17, 2013 at 8:57 PM, shashwat shriparv < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Ye its is asking for file:/// instead of hdfs:// just check if it >>>>>>>>>> is taking setting configuration from other location... >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> ∞ >>>>>>>>>> Shashwat Shriparv >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Sun, Mar 17, 2013 at 11:07 PM, Luangsay Sourygna < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> What is the version of Hadoop you use? >>>>>>>>>>> >>>>>>>>>>> Try using fs.defaultFS instead of fs.default.name (see the list >>>>>>>>>>> of all >>>>>>>>>>> the deprecated properties here: >>>>>>>>>>> >>>>>>>>>>> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/DeprecatedProperties.html >>>>>>>>>>> ). >>>>>>>>>>> I remember I once had a similar error message and it was due to >>>>>>>>>>> the >>>>>>>>>>> change in properties names. >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> >>>>>>>>>>> Sourygna >>>>>>>>>>> >>>>>>>>>>> On Sun, Mar 17, 2013 at 2:32 PM, Mohammad Alkahtani >>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>> > Hi to all users of Hadoop, >>>>>>>>>>> > >>>>>>>>>>> > I installed Hadoop the .deb file on Ubuntu 12.04 but I might >>>>>>>>>>> could not >>>>>>>>>>> > configure it right. The conf dir is under templates in >>>>>>>>>>> /usr/shar/hadoop. I >>>>>>>>>>> > edit the core-site.xml, mapred-site.xml files to give >>>>>>>>>>> > <property> >>>>>>>>>>> > <name>fs.default.name</name> >>>>>>>>>>> > <value>hdfs://localhost:9000</value> >>>>>>>>>>> > </property> >>>>>>>>>>> > and for mapred >>>>>>>>>>> > <property> >>>>>>>>>>> > <name>mapred.job.tracker</name> >>>>>>>>>>> > <value>localhost:9001</value> >>>>>>>>>>> > </property> >>>>>>>>>>> > >>>>>>>>>>> > but i get these errors, I assume that there is problem, Hadoop >>>>>>>>>>> cannot read >>>>>>>>>>> > the configuration file. >>>>>>>>>>> > I chaned the hadoop-env.sh to >>>>>>>>>>> > export >>>>>>>>>>> HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/usr/shar/hadoop/templates/conf"} >>>>>>>>>>> > but dosen't solve the problem. >>>>>>>>>>> > >>>>>>>>>>> > ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: >>>>>>>>>>> > java.lang.IllegalArgumentException: Does not contain a valid >>>>>>>>>>> host:port >>>>>>>>>>> > authority: file:/// at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164) >>>>>>>>>>> at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:201) >>>>>>>>>>> > at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:231) >>>>>>>>>>> > at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getServiceAddress(NameNode.java:225) >>>>>>>>>>> > at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:347) >>>>>>>>>>> > at >>>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.(DataNode.java:309) >>>>>>>>>>> at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1651) >>>>>>>>>>> > at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1590) >>>>>>>>>>> > at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1608) >>>>>>>>>>> > at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1734) >>>>>>>>>>> > at >>>>>>>>>>> org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1751) >>>>>>>>>>> > >>>>>>>>>>> > ________________________________ >>>>>>>>>>> > >>>>>>>>>>> > FATAL org.apache.hadoop.mapred.JobTracker: >>>>>>>>>>> > java.lang.IllegalArgumentException: Does not contain a valid >>>>>>>>>>> host:port >>>>>>>>>>> > authority: local at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164) >>>>>>>>>>> at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:130) >>>>>>>>>>> at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:2312) >>>>>>>>>>> at >>>>>>>>>>> > org.apache.hadoop.mapred.JobTracker.(JobTracker.java:2070) at >>>>>>>>>>> > org.apache.hadoop.mapred.JobTracker.(JobTracker.java:1889) at >>>>>>>>>>> > org.apache.hadoop.mapred.JobTracker.(JobTracker.java:1883) at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:312) >>>>>>>>>>> at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:303) >>>>>>>>>>> at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:298) >>>>>>>>>>> > at >>>>>>>>>>> org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4791) >>>>>>>>>>> > >>>>>>>>>>> > ________________________________ >>>>>>>>>>> > >>>>>>>>>>> > ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: >>>>>>>>>>> > java.lang.IllegalArgumentException: Does not contain a valid >>>>>>>>>>> host:port >>>>>>>>>>> > authority: file:/// at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164) >>>>>>>>>>> at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:201) >>>>>>>>>>> > at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:231) >>>>>>>>>>> > at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:265) >>>>>>>>>>> > at >>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:536) >>>>>>>>>>> at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1410) >>>>>>>>>>> > at >>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1419) >>>>>>>>>>> > >>>>>>>>>>> > ________________________________ >>>>>>>>>>> > >>>>>>>>>>> > Exception in thread "main" java.lang.IllegalArgumentException: >>>>>>>>>>> Does not >>>>>>>>>>> > contain a valid host:port authority: file:/// at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164) >>>>>>>>>>> at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:201) >>>>>>>>>>> > at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:231) >>>>>>>>>>> > at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getServiceAddress(NameNode.java:225) >>>>>>>>>>> > at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.initialize(SecondaryNameNode.java:167) >>>>>>>>>>> > at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.(SecondaryNameNode.java:135) >>>>>>>>>>> > at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.main(SecondaryNameNode.java:650) >>>>>>>>>>> > >>>>>>>>>>> > ________________________________ >>>>>>>>>>> > >>>>>>>>>>> > ERROR org.apache.hadoop.mapred.TaskTracker: Can not start task >>>>>>>>>>> tracker >>>>>>>>>>> > because java.lang.IllegalArgumentException: Does not contain a >>>>>>>>>>> valid >>>>>>>>>>> > host:port authority: local at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164) >>>>>>>>>>> at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:130) >>>>>>>>>>> at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:2312) >>>>>>>>>>> at >>>>>>>>>>> > org.apache.hadoop.mapred.TaskTracker.(TaskTracker.java:1532) at >>>>>>>>>>> > >>>>>>>>>>> org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3906) >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > Regards, >>>>>>>>>>> > Mohammad Alkahtani >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
