[ https://issues.apache.org/jira/browse/SPARK-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dongxu updated SPARK-5527: -------------------------- Description: We must make all node conf file consistent when we start our standalone cluster. For example, we set "SPARK_WORKER_INSTANCES=2" to start 2 worker on each machine. I see this code at $SPARK_HOME/sbin/spark-daemon.sh if [ "$SPARK_MASTER" != "" ]; then echo rsync from $SPARK_MASTER rsync -a -e ssh --delete --exclude=.svn --exclude='logs/*' --exclude='contrib/hod/logs/*' $SPARK_MASTER/ "$SPARK_HOME" fi I think we better mention it at document . was: We need make all node conf file consistency when we start our standalone cluster. For example, we set "SPARK_WORKER_INSTANCES=2" to start 2 worker on each machine. I see this code at $SPARK_HOME/sbin/spark-daemon.sh if [ "$SPARK_MASTER" != "" ]; then echo rsync from $SPARK_MASTER rsync -a -e ssh --delete --exclude=.svn --exclude='logs/*' --exclude='contrib/hod/logs/*' $SPARK_MASTER/ "$SPARK_HOME" fi I think we better mention it at document . > Add standalone document configiration to explain how to make cluster conf > file consistency > ------------------------------------------------------------------------------------------- > > Key: SPARK-5527 > URL: https://issues.apache.org/jira/browse/SPARK-5527 > Project: Spark > Issue Type: Documentation > Components: Documentation > Affects Versions: 1.2.0, 1.3.0 > Reporter: dongxu > Priority: Minor > Labels: docuentation, starter > Original Estimate: 10m > Remaining Estimate: 10m > > We must make all node conf file consistent when we start our standalone > cluster. For example, we set "SPARK_WORKER_INSTANCES=2" to start 2 worker > on each machine. > I see this code at $SPARK_HOME/sbin/spark-daemon.sh > if [ "$SPARK_MASTER" != "" ]; then > echo rsync from $SPARK_MASTER > rsync -a -e ssh --delete --exclude=.svn --exclude='logs/*' > --exclude='contrib/hod/logs/*' $SPARK_MASTER/ "$SPARK_HOME" > fi > I think we better mention it at document . -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org