Jeff Hubbs created HDFS-13396:

             Summary: and has malformed command; 
doesn't use workers file
                 Key: HDFS-13396
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: hdfs
    Affects Versions: 3.0.1, 3.0.0
         Environment: Hadoop 3.0.1 binary distribution on Gentoo Linux, Icedtea 
            Reporter: Jeff Hubbs

In 3.0.1's, the command to start the datanodes reads as follows:
hadoop_uservar_su hdfs datanode "${HADOOP_HDFS_HOME}/bin/hdfs" \
    --workers \
    --config "${HADOOP_CONF_DIR}" \
    --daemon start \
    datanode ${dataStartOpt}
 This doesn't work; executing the script produces this:
hdfs@msba02a ~ $ $HADOOP_HOME/sbin/
Starting namenodes on [msba02a.bus.emory.ddns]
Starting datanodes
^/opt/hadoop/3: ssh: Could not resolve hostname 
^/opt/hadoop/3.0.1/etc/hadoop/workers: Name or service not known
pdsh@msba02a: ^/opt/hadoop/3: ssh exited with exit code 255
Starting secondary namenodes [msba02a]

It misinterprets the value of HADOOP_CONF_DIR as one of the names of a machine 
it is supposed to access.

The workaround I developed involves the --hostnames option like so, changing 
the one-name-per-line workers file into a comma-separated list:
hadoop_uservar_su hdfs datanode "${HADOOP_HDFS_HOME}/bin/hdfs" \
    --workers \
    --hostnames `sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/,/g' 
${HADOOP_CONF_DIR}/workers` \
    --config "${HADOOP_CONF_DIR}" \
    --daemon start \
    datanode ${dataStartOpt}

A similar change had to be made to I've verified that 
HADOOP_HDFS_HOME and HADOOP_CONF_DIR are set correctly within the script at the 
point where this command executes.

This problem also exists in in 3.0.0, although the 
original invocation differs slightly from 3.0.1.

In 3.0.1, I'm running into another problem with getting datanodes started (was 
fine in 3.0.0) but I couldn't hit that problem until I got past this one.


This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to