Hi Rohith,
Do you have multiple interfaces on the machine hosting the master ?
If so, can you try to force to the public interface using:
sbin/start-master.sh --ip xxx.xxx.xxx.xxx
Regards
JB
On 10/19/2015 02:05 PM, Rohith Parameshwara wrote:
Hi all,
I am doing some experiments on spark standalone cluster
setup and I am facing the following issue:
I have a 4 node cluster setup. As per
http://spark.apache.org/docs/latest/spark-standalone.html#starting-a-cluster-manually
I tried to start the cluster with the scripts but, the slaves did not
start and gave permission denied error….My conf/slaves had a list of IP
addresses of the slaves… But Then I was able to start the worker nodes
by going to the respective slave machine and running the start-slave.sh
script with the master IP as parameter…. The webui showed the all the 3
worker nodes running as :
The application when submitted to the cluster will show in all the work
nodes in the webui…. But in the command line where is spark-submit
command was run, it gives this error periodically and continuously….
ERROR NettyTransport: failed to bind to /172.28.161.138:7077, shutting
down Netty transport
15/10/19 11:59:38 WARN Utils: Service 'sparkDriver' could not bind on
port 7077. Attempting port 7078.
15/10/19 11:59:38 WARN AppClient$ClientActor: Could not connect to
akka.tcp://sparkMaster@localhost:7077: akka.remote.InvalidAssociation:
Invalid address: akka.tcp://sparkMaster@localhost:7077….
My conf/spark-env.sh has:
SPARK_MASTER_IP=172.28.161.138
SPARK_MASTER_PORT=7077
SPARK_MASTER_WEBUI_PORT=8080
SPARK_WORKER_WEBUI_PORT=8081
SPARK_WORKER_INSTANCES=1
And I have also put this in all the slave nodes too….
The applications are running fine in –master local mode but in –master
spark://masterip:7077, it is not working….
Any type of help would be appreciated….. Thanks in advance
--
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org
