Jeff Hubbs created HDFS-13396: --------------------------------- Summary: start-dfs.sh and stop-dfs.sh has malformed command; doesn't use workers file Key: HDFS-13396 URL: https://issues.apache.org/jira/browse/HDFS-13396 Project: Hadoop HDFS Issue Type: Bug Components: hdfs Affects Versions: 3.0.1, 3.0.0 Environment: Hadoop 3.0.1 binary distribution on Gentoo Linux, Icedtea JRE Reporter: Jeff Hubbs
In 3.0.1's start-dfs.sh, the command to start the datanodes reads as follows: {code:java} hadoop_uservar_su hdfs datanode "${HADOOP_HDFS_HOME}/bin/hdfs" \ --workers \ --config "${HADOOP_CONF_DIR}" \ --daemon start \ datanode ${dataStartOpt} {code} This doesn't work; executing the script produces this: {code:java} hdfs@msba02a ~ $ $HADOOP_HOME/sbin/start-dfs.sh Starting namenodes on [msba02a.bus.emory.ddns] Starting datanodes ^/opt/hadoop/3: ssh: Could not resolve hostname ^/opt/hadoop/3.0.1/etc/hadoop/workers: Name or service not known pdsh@msba02a: ^/opt/hadoop/3: ssh exited with exit code 255 Starting secondary namenodes [msba02a] {code} It misinterprets the value of HADOOP_CONF_DIR as one of the names of a machine it is supposed to access. The workaround I developed involves the --hostnames option like so, changing the one-name-per-line workers file into a comma-separated list: {code:java} hadoop_uservar_su hdfs datanode "${HADOOP_HDFS_HOME}/bin/hdfs" \ --workers \ --hostnames `sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/,/g' ${HADOOP_CONF_DIR}/workers` \ --config "${HADOOP_CONF_DIR}" \ --daemon start \ datanode ${dataStartOpt} {code} A similar change had to be made to stop-dfs.sh. I've verified that HADOOP_HDFS_HOME and HADOOP_CONF_DIR are set correctly within the script at the point where this command executes. This problem also exists in start-dfs.sh/stop-dfs.sh in 3.0.0, although the original invocation differs slightly from 3.0.1. In 3.0.1, I'm running into another problem with getting datanodes started (was fine in 3.0.0) but I couldn't hit that problem until I got past this one. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org