Yes, as Leo said. Be sure that you deploy all config files around your cluster. and be sure that you installed the same hadoop version. In my blog you'll find a older page "running an hadoop cluster in 20 minutes", based on a distribution, but the main steps are the same.
best, Alex -- Alexander Lorenz http://mapredit.blogspot.com German Hadoop LinkedIn Group: http://goo.gl/N8pCF On Apr 27, 2012, at 7:53 PM, Leo Leung wrote: > I would suggest changing the hadoop configuration on the slave(s) > You'll have to maintain the delta config amongst the nodes. > > > -----Original Message----- > From: Björn-Elmar Macek [mailto:ma...@cs.uni-kassel.de] > Sent: Friday, April 27, 2012 7:28 AM > To: hdfs-user@hadoop.apache.org > Subject: Re: Hadoop Configuration Issues > > Hi Alex, > > thank you for the tip: it pushed me into the right direction. I was using the > deb-package to install hadoop, which did not work out cause of the problems i > had. Now i use the tarball archive and unpacked it to a subfolder of my home > directory. > > BUT i just ran into another problem: after successfully executing "hadoop > namenode -format" the "start-all.sh" script (executed on the master server) > runs into errors, when trying to access files on the slave system: it > automatically seems to expect the hadoop files lying in the very same > directory as on the master: > > ema@ubuntu:~/Programs/hadoop-1.0.2/bin$ ./start-all.sh > Warning: $HADOOP_HOME is deprecated. > > namenode running as process 19393. Stop it first. > slave: bash: line 0: cd: /home/ema/Programs/hadoop-1.0.2/libexec/..: No such > file or directory > slave: bash: /home/ema/Programs/hadoop-1.0.2/bin/hadoop-daemon.sh: No such > file or directory ema@master's password: > master: Connection closed by UNKNOWN > starting jobtracker, logging to > /var/log/hadoop/ema/hadoop-ema-jobtracker-ubuntu.out > slave: bash: line 0: cd: /home/ema/Programs/hadoop-1.0.2/libexec/..: No such > file or directory > slave: bash: /home/ema/Programs/hadoop-1.0.2/bin/hadoop-daemon.sh: No such > file or directory > > How can i tell the slave, that the files lie somewhere else? > > Best regards, > Björn-Elmar > > Am 27.04.2012 13:11, schrieb alo alt: >> Hi, >> >> yes, sorry - I saw that after I hit the send botton. >> Looks like you mixed up some configs with wrong templates. I would suggest >> you use the default configs: >> http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-common-project >> /hadoop-common/src/main/conf/ >> >> - Alex >> >> -- >> Alexander Lorenz >> http://mapredit.blogspot.com >> >> On Apr 27, 2012, at 12:39 PM, Björn-Elmar Macek wrote: >> >>> Hi Alex, >>> >>> as i have written, i already did so! The problem is as already stated in my >>> mail before, that all the Variables ${bla} seem to be UNSET - not only >>> SECURITY_TYPE. As i dont really understand those parameters, i would like >>> to use the default ones, which afaik should be configured in the >>> hadoop-env.sh. But obviously they are not. >>> >>> Best, >>> Björn >>> >>> Am 27.04.2012 12:12, schrieb alo alt: >>>> Hi, >>>> >>>> Invalid attribute value for hadoop.security.authentication of >>>> ${SECURITY_TYPE} Set it to simple and it should work (default is kerberos). >>>> >>>> - Alex >>>> >>>> -- >>>> Alexander Lorenz >>>> http://mapredit.blogspot.com >>>> >>>> On Apr 27, 2012, at 12:01 PM, Björn-Elmar Macek wrote: >>>> >>>>> Hello, >>>>> >>>>> i have recently installed Hadoop on my and a second machine in order to >>>>> test the setup and develop little programs locally before deploying them >>>>> to the cluster. I stumbled over several difficulties, which i could fix >>>>> with some internet research. But once again im stuck and i think i can >>>>> nail the problem down: >>>>> >>>>> When Hadoop evaluates the config files in /etc/hadoop it does not have >>>>> any default values for all the variables used within: >>>>> >>>>> \________ First Error: >>>>> hadoop namenode -format >>>>> Warning: $HADOOP_HOME is deprecated. >>>>> >>>>> 12/04/27 11:31:41 INFO namenode.NameNode: STARTUP_MSG: >>>>> /************************************************************ >>>>> STARTUP_MSG: Starting NameNode >>>>> STARTUP_MSG: host = ubuntu/127.0.1.1 >>>>> STARTUP_MSG: args = [-format] >>>>> STARTUP_MSG: version = 1.0.1 >>>>> STARTUP_MSG: build = >>>>> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r >>>>> 1243785; compiled by 'hortonfo' on Tue Feb 14 08:13:52 UTC 2012 >>>>> ************************************************************/ >>>>> 12/04/27 11:31:41 INFO util.GSet: VM type = 32-bit >>>>> 12/04/27 11:31:41 INFO util.GSet: 2% max memory = 2.475 MB >>>>> 12/04/27 11:31:41 INFO util.GSet: capacity = 2^19 = 524288 entries >>>>> 12/04/27 11:31:41 INFO util.GSet: recommended=524288, actual=524288 >>>>> 12/04/27 11:31:41 ERROR namenode.NameNode: >>>>> java.lang.IllegalArgumentException: Invalid attribute value for >>>>> hadoop.security.authentication of ${SECURITY_TYPE} >>>>> at >>>>> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:202) >>>>> at >>>>> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:187) >>>>> at >>>>> org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:239) >>>>> at >>>>> org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:438) >>>>> at >>>>> org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:424) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setConfigurationParameters(FSNamesystem.java:473) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:462) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1162) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1271) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java: >>>>> 1288) >>>>> >>>>> 12/04/27 11:31:41 INFO namenode.NameNode: SHUTDOWN_MSG: >>>>> /************************************************************ >>>>> SHUTDOWN_MSG: Shutting down NameNode at ubuntu/127.0.1.1 >>>>> ************************************************************/ >>>>> >>>>> >>>>> \_________ Solution >>>>> I removed the variable and exchanged it with the value "simple". >>>>> Then the next error occurred: >>>>> >>>>> >>>>> \_________ Error 2 >>>>> hadoop namenode -format >>>>> Warning: $HADOOP_HOME is deprecated. >>>>> >>>>> 12/04/27 11:46:33 INFO namenode.NameNode: STARTUP_MSG: >>>>> /************************************************************ >>>>> STARTUP_MSG: Starting NameNode >>>>> STARTUP_MSG: host = ubuntu/127.0.1.1 >>>>> STARTUP_MSG: args = [-format] >>>>> STARTUP_MSG: version = 1.0.1 >>>>> STARTUP_MSG: build = >>>>> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r >>>>> 1243785; compiled by 'hortonfo' on Tue Feb 14 08:13:52 UTC 2012 >>>>> ************************************************************/ >>>>> 12/04/27 11:46:33 INFO util.GSet: VM type = 32-bit >>>>> 12/04/27 11:46:33 INFO util.GSet: 2% max memory = 2.475 MB >>>>> 12/04/27 11:46:33 INFO util.GSet: capacity = 2^19 = 524288 entries >>>>> 12/04/27 11:46:33 INFO util.GSet: recommended=524288, actual=524288 >>>>> 12/04/27 11:46:33 ERROR namenode.NameNode: >>>>> java.lang.ExceptionInInitializerError >>>>> at >>>>> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:212) >>>>> at >>>>> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:187) >>>>> at >>>>> org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:239) >>>>> at >>>>> org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:438) >>>>> at >>>>> org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:424) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setConfigurationParameters(FSNamesystem.java:473) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:462) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1162) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1271) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java: >>>>> 1288) Caused by: java.util.regex.PatternSyntaxException: Illegal >>>>> repetition near index 8 [jt]t@.*${KERBEROS_REALM} >>>>> ^ >>>>> at java.util.regex.Pattern.error(Pattern.java:1730) >>>>> at java.util.regex.Pattern.closure(Pattern.java:2792) >>>>> at java.util.regex.Pattern.sequence(Pattern.java:1906) >>>>> at java.util.regex.Pattern.expr(Pattern.java:1769) >>>>> at java.util.regex.Pattern.compile(Pattern.java:1477) >>>>> at java.util.regex.Pattern.<init>(Pattern.java:1150) >>>>> at java.util.regex.Pattern.compile(Pattern.java:840) >>>>> at >>>>> org.apache.hadoop.security.KerberosName$Rule.<init>(KerberosName.java:188) >>>>> at >>>>> org.apache.hadoop.security.KerberosName.parseRules(KerberosName.java:324) >>>>> at >>>>> org.apache.hadoop.security.KerberosName.setConfiguration(KerberosName.java:343) >>>>> at >>>>> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:212) >>>>> at >>>>> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:187) >>>>> at >>>>> org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:239) >>>>> at >>>>> org.apache.hadoop.security.KerberosName.<clinit>(KerberosName.java:83) >>>>> ... 10 more >>>>> >>>>> 12/04/27 11:46:33 INFO namenode.NameNode: SHUTDOWN_MSG: >>>>> /************************************************************ >>>>> SHUTDOWN_MSG: Shutting down NameNode at ubuntu/127.0.1.1 >>>>> ************************************************************/ >>>>> >>>>> \______ And once again... >>>>> ... a variable seems to be undefined. And i guess, if i found a suitable >>>>> value for this property the next one will be undefined. I hope there are >>>>> any default values present, because i have no idea, what to fill to the >>>>> value slots. All books and instructions i read on hadoop never discussed >>>>> these issues. >>>>> BTW: HADOOP_HOME is defined, although the log tells different. >>>>> >>>>> I hope you can assist me. >>>>> >>>>> Best regards, >>>>> Björn-Elmar Macek >>>>> >> >
