Hi Alex,
thank you for the tip: it pushed me into the right direction. I was
using the deb-package to install hadoop, which did not work out cause of
the problems i had. Now i use the tarball archive and unpacked it to a
subfolder of my home directory.
BUT i just ran into another problem: after successfully executing
"hadoop namenode -format" the "start-all.sh" script (executed on the
master server) runs into errors, when trying to access files on the
slave system: it automatically seems to expect the hadoop files lying in
the very same directory as on the master:
ema@ubuntu:~/Programs/hadoop-1.0.2/bin$ ./start-all.sh
Warning: $HADOOP_HOME is deprecated.
namenode running as process 19393. Stop it first.
slave: bash: line 0: cd: /home/ema/Programs/hadoop-1.0.2/libexec/..: No
such file or directory
slave: bash: /home/ema/Programs/hadoop-1.0.2/bin/hadoop-daemon.sh: No
such file or directory
ema@master's password:
master: Connection closed by UNKNOWN
starting jobtracker, logging to
/var/log/hadoop/ema/hadoop-ema-jobtracker-ubuntu.out
slave: bash: line 0: cd: /home/ema/Programs/hadoop-1.0.2/libexec/..: No
such file or directory
slave: bash: /home/ema/Programs/hadoop-1.0.2/bin/hadoop-daemon.sh: No
such file or directory
How can i tell the slave, that the files lie somewhere else?
Best regards,
Björn-Elmar
Am 27.04.2012 13:11, schrieb alo alt:
Hi,
yes, sorry - I saw that after I hit the send botton.
Looks like you mixed up some configs with wrong templates. I would suggest you
use the default configs:
http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/
- Alex
--
Alexander Lorenz
http://mapredit.blogspot.com
On Apr 27, 2012, at 12:39 PM, Björn-Elmar Macek wrote:
Hi Alex,
as i have written, i already did so! The problem is as already stated in my
mail before, that all the Variables ${bla} seem to be UNSET - not only
SECURITY_TYPE. As i dont really understand those parameters, i would like to
use the default ones, which afaik should be configured in the hadoop-env.sh.
But obviously they are not.
Best,
Björn
Am 27.04.2012 12:12, schrieb alo alt:
Hi,
Invalid attribute value for hadoop.security.authentication of ${SECURITY_TYPE}
Set it to simple and it should work (default is kerberos).
- Alex
--
Alexander Lorenz
http://mapredit.blogspot.com
On Apr 27, 2012, at 12:01 PM, Björn-Elmar Macek wrote:
Hello,
i have recently installed Hadoop on my and a second machine in order to test
the setup and develop little programs locally before deploying them to the
cluster. I stumbled over several difficulties, which i could fix with some
internet research. But once again im stuck and i think i can nail the problem
down:
When Hadoop evaluates the config files in /etc/hadoop it does not have any
default values for all the variables used within:
\________ First Error:
hadoop namenode -format
Warning: $HADOOP_HOME is deprecated.
12/04/27 11:31:41 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = ubuntu/127.0.1.1
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 1.0.1
STARTUP_MSG: build =
https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1243785;
compiled by 'hortonfo' on Tue Feb 14 08:13:52 UTC 2012
************************************************************/
12/04/27 11:31:41 INFO util.GSet: VM type = 32-bit
12/04/27 11:31:41 INFO util.GSet: 2% max memory = 2.475 MB
12/04/27 11:31:41 INFO util.GSet: capacity = 2^19 = 524288 entries
12/04/27 11:31:41 INFO util.GSet: recommended=524288, actual=524288
12/04/27 11:31:41 ERROR namenode.NameNode: java.lang.IllegalArgumentException:
Invalid attribute value for hadoop.security.authentication of ${SECURITY_TYPE}
at
org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:202)
at
org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:187)
at
org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:239)
at
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:438)
at
org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:424)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setConfigurationParameters(FSNamesystem.java:473)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:462)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1162)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1271)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)
12/04/27 11:31:41 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at ubuntu/127.0.1.1
************************************************************/
\_________ Solution
I removed the variable and exchanged it with the value "simple".
Then the next error occurred:
\_________ Error 2
hadoop namenode -format
Warning: $HADOOP_HOME is deprecated.
12/04/27 11:46:33 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = ubuntu/127.0.1.1
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 1.0.1
STARTUP_MSG: build =
https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1243785;
compiled by 'hortonfo' on Tue Feb 14 08:13:52 UTC 2012
************************************************************/
12/04/27 11:46:33 INFO util.GSet: VM type = 32-bit
12/04/27 11:46:33 INFO util.GSet: 2% max memory = 2.475 MB
12/04/27 11:46:33 INFO util.GSet: capacity = 2^19 = 524288 entries
12/04/27 11:46:33 INFO util.GSet: recommended=524288, actual=524288
12/04/27 11:46:33 ERROR namenode.NameNode: java.lang.ExceptionInInitializerError
at
org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:212)
at
org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:187)
at
org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:239)
at
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:438)
at
org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:424)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setConfigurationParameters(FSNamesystem.java:473)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:462)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1162)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1271)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)
Caused by: java.util.regex.PatternSyntaxException: Illegal repetition near
index 8
[jt]t@.*${KERBEROS_REALM}
^
at java.util.regex.Pattern.error(Pattern.java:1730)
at java.util.regex.Pattern.closure(Pattern.java:2792)
at java.util.regex.Pattern.sequence(Pattern.java:1906)
at java.util.regex.Pattern.expr(Pattern.java:1769)
at java.util.regex.Pattern.compile(Pattern.java:1477)
at java.util.regex.Pattern.<init>(Pattern.java:1150)
at java.util.regex.Pattern.compile(Pattern.java:840)
at
org.apache.hadoop.security.KerberosName$Rule.<init>(KerberosName.java:188)
at org.apache.hadoop.security.KerberosName.parseRules(KerberosName.java:324)
at
org.apache.hadoop.security.KerberosName.setConfiguration(KerberosName.java:343)
at
org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:212)
at
org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:187)
at
org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:239)
at org.apache.hadoop.security.KerberosName.<clinit>(KerberosName.java:83)
... 10 more
12/04/27 11:46:33 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at ubuntu/127.0.1.1
************************************************************/
\______ And once again...
... a variable seems to be undefined. And i guess, if i found a suitable value
for this property the next one will be undefined. I hope there are any default
values present, because i have no idea, what to fill to the value slots. All
books and instructions i read on hadoop never discussed these issues.
BTW: HADOOP_HOME is defined, although the log tells different.
I hope you can assist me.
Best regards,
Björn-Elmar Macek
