there is docker script for spark 0.9 in spark git Yours, Xuefeng Wu 吴雪峰 敬上
> On 2014年8月10日, at 下午8:27, 诺铁 <noty...@gmail.com> wrote: > > hi, all, > > I am playing with docker, trying to create a spark cluster with docker > containers. > > since spark master, worker, driver all need to visit each other, I configured > a dns server, and set hostname and domain name of each node. but when spark > master start up, it seems to be using hostname without domain name. > > I see lines in sbin/start-master.sh > if [ "$SPARK_MASTER_IP" = "" ]; then > SPARK_MASTER_IP=`hostname` > fi > > and when trying to use spark-shell, it seems also listen to hostname, and > worker failed to connect to driver by hostname. > > I manually set > export SPARK_MASTER_IP=my.master.domain.name > export SPARK_LOCAL_IP=my.master.domain.name > > everything works fine. > > so I am thinking where can I change this, I found SPARK_PUBLIC_DNS in spark > configuration, but document is very simple, I don't know what it do, and what > value should I set. > > should I use this setting and how? > > thanks > > >
