Jeff Hubbs created HDFS-13396:
---------------------------------
Summary: start-dfs.sh and stop-dfs.sh has malformed command;
doesn't use workers file
Key: HDFS-13396
URL: https://issues.apache.org/jira/browse/HDFS-13396
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs
Affects Versions: 3.0.1, 3.0.0
Environment: Hadoop 3.0.1 binary distribution on Gentoo Linux, Icedtea
JRE
Reporter: Jeff Hubbs
In 3.0.1's start-dfs.sh, the command to start the datanodes reads as follows:
{code:java}
hadoop_uservar_su hdfs datanode "${HADOOP_HDFS_HOME}/bin/hdfs" \
--workers \
--config "${HADOOP_CONF_DIR}" \
--daemon start \
datanode ${dataStartOpt}
{code}
This doesn't work; executing the script produces this:
{code:java}
hdfs@msba02a ~ $ $HADOOP_HOME/sbin/start-dfs.sh
Starting namenodes on [msba02a.bus.emory.ddns]
Starting datanodes
^/opt/hadoop/3: ssh: Could not resolve hostname
^/opt/hadoop/3.0.1/etc/hadoop/workers: Name or service not known
pdsh@msba02a: ^/opt/hadoop/3: ssh exited with exit code 255
Starting secondary namenodes [msba02a]
{code}
It misinterprets the value of HADOOP_CONF_DIR as one of the names of a machine
it is supposed to access.
The workaround I developed involves the --hostnames option like so, changing
the one-name-per-line workers file into a comma-separated list:
{code:java}
hadoop_uservar_su hdfs datanode "${HADOOP_HDFS_HOME}/bin/hdfs" \
--workers \
--hostnames `sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/,/g'
${HADOOP_CONF_DIR}/workers` \
--config "${HADOOP_CONF_DIR}" \
--daemon start \
datanode ${dataStartOpt}
{code}
A similar change had to be made to stop-dfs.sh. I've verified that
HADOOP_HDFS_HOME and HADOOP_CONF_DIR are set correctly within the script at the
point where this command executes.
This problem also exists in start-dfs.sh/stop-dfs.sh in 3.0.0, although the
original invocation differs slightly from 3.0.1.
In 3.0.1, I'm running into another problem with getting datanodes started (was
fine in 3.0.0) but I couldn't hit that problem until I got past this one.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
