Hi Andrew, I do most of my Hadoop development on MacOS and I've always wondered about that message. I tried your fix and it works.
Thanks! Geoff On Mar 12, 2014, at 9:40 AM, Andrew Pennebaker <[email protected]> wrote: > In recent versions of Mac OS X, a default Hadoop configuration such as from > Homebrew raises errors on some operations: > > $ hadoop version > Hadoop 1.2.1 > Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 > -r 1503152 > Compiled by mattf on Mon Jul 22 15:23:09 PDT 2013 > From source with checksum 6923c86528809c4e7e6f493b6b413a9a > This command was run using > /usr/local/Cellar/hadoop/1.2.1/libexec/hadoop-core-1.2.1.jar > > When querying HDFS, Hadoop reports a realm error: > > $ hadoop fsck / -files -bytes > 2014-03-12 10:55:48.330 java[12749:1703] Unable to load realm info from > SCDynamicStore > ... > > This happens because realms are not configured by default. Setting > HADOOP_OPTS in hadoop-env.sh fixes the realm error: > > export HADOOP_OPTS="-Djava.security.krb5.realm=-Djava.security.krb5.kdc=" > > $ hadoop fsck / -files -bytes > ... > > Another error occurs when attempting to start a namenode: > > $ hadoop namenode > 14/03/12 11:25:25 ERROR namenode.NameNode: > java.lang.IllegalArgumentException: Does not contain a valid host:port > authority: file:/// > > This is because the core-site.xml bundled with Hadoop fails to specify an > address. Adding an address property fixes this: > > <property> > <name>fs.default.name</name> > <value>hdfs://localhost:8020</value> > </property> > > $ hadoop namenode > ... > 14/03/12 11:27:53 INFO ipc.Server: IPC Server listener on 8020: starting > > These manual steps to correct the configuration might scare off newbies. In > the future, could Hadoop come with better defaults, for a better out of the > box experience for newbies? > > -- > Cheers, > > Andrew Pennebaker > [email protected]
