When I tried starting hadoop using the init scripts provided, the master couldn't find any of the datanodes. It is my understanding that the masters file is optional, but the slaves file is required. The scripts that reference the slaves file are named in plural (instead of hadoop-daemon.sh, use hadoop-daemons.sh). I tried modifying the init scripts to run hadoop-daemons.sh, and the script attempted to spawn processes on the slaves referenced in the slaves file, but that produced the error: Starting Hadoop namenode: [ OK ] slave2: starting namenode, logging to /usr/lib/hadoop/logs/hadoop-hadoopuser-namenode-loki.out master: starting namenode, logging to /usr/lib/hadoop/logs/hadoop-hadoopuser-namenode-odin.out slave3: starting namenode, logging to /usr/lib/hadoop/logs/hadoop-hadoopuser-namenode-tyr.out slave1: starting namenode, logging to /usr/lib/hadoop/logs/hadoop-hadoopuser-namenode-thor.out slave2: /usr/lib/hadoop-hdfs/bin/hdfs: line 34: /usr/lib/hadoop-hdfs/bin/../libexec/hdfs-config.sh: No such file or directory slave2: /usr/lib/hadoop-hdfs/bin/hdfs: line 150: cygpath: command not found slave2: /usr/lib/hadoop-hdfs/bin/hdfs: line 191: exec: : not found slave3: /usr/lib/hadoop-hdfs/bin/hdfs: line 34: /usr/lib/hadoop-hdfs/bin/../libexec/hdfs-config.sh: No such file or directory slave3: /usr/lib/hadoop-hdfs/bin/hdfs: line 150: cygpath: command not found slave3: /usr/lib/hadoop-hdfs/bin/hdfs: line 191: exec: : not found master: /usr/lib/hadoop-hdfs/bin/hdfs: line 34: /usr/lib/hadoop-hdfs/bin/../libexec/hdfs-config.sh: No such file or directory master: /usr/lib/hadoop-hdfs/bin/hdfs: line 150: cygpath: command not found master: /usr/lib/hadoop-hdfs/bin/hdfs: line 191: exec: : not found slave1: /usr/lib/hadoop-hdfs/bin/hdfs: line 34: /usr/lib/hadoop-hdfs/bin/../libexec/hdfs-config.sh: No such file or directory slave1: /usr/lib/hadoop-hdfs/bin/hdfs: line 150: cygpath: command not found slave1: /usr/lib/hadoop-hdfs/bin/hdfs: line 191: exec: : not found
-David Fryer On Mon, Jul 21, 2014 at 12:18 PM, Konstantin Boudnik <[email protected]> wrote: > Hi David. > > Slaves files are really optional if I remember right. In Bigtop we are > usually > deploy Hadoop with provided Puppet recipes which are battle-hardened over > the > years :) > > Cos > > On Mon, Jul 21, 2014 at 10:53AM, David Fryer wrote: > > Hi Bigtop! > > > > I'm working on trying to get hadoop running in distributed mode, but the > > init scripts don't seem to be referencing the slaves file in > > /etc/hadoop/conf. Has anyone encountered this before? > > > > Thanks, > > David Fryer >
