Re: Spark in cluster [ remote.EndpointWriter: AssociationError]
Sorry, but your solution doesn't work. I can see on my master port 7077 open and listening and connected workers but I don't understand why it's trying to connect itself ... => Master is running on the specific host netstat -at | grep 7077 You will get something similar to: tcp0 0 akhldz.master.io:7077 *:* LISTEN If that is the case, then from your worker machine do a host akhldz.master.io ( replace akhldz.master.io with your master host. If something goes wrong, then add a host entry in your /etc/hosts file) telnet akhldz.master.io 7077 ( If this is not connecting, then your worker wont connect either. ) => Adding Host entry in /etc/hosts Open /etc/hosts from your worker machine and add the following entry (example) 192.168.100.20 akhldz.master.io PS :In the above case Pillis was having two ip addresses having same host name eg: 192.168.100.40 s1.machine.org 192.168.100.41 s1.machine.org -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-in-cluster-remote-EndpointWriter-AssociationError-tp16063p16134.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: Spark in cluster [ remote.EndpointWriter: AssociationError]
Can you paste your spark-env.sh file? Looks like you have it misconfigured. Thanks Best Regards On Fri, Oct 10, 2014 at 1:43 AM, Morbious wrote: > Hi, > > Recently I've configured spark in cluster with zookeper. > I have 2 masters ( active/standby) and 6 workers. > I've begun my installation with samples from example directory. > Everything worked fine when I only used memory . > When I used word count example I got messages like the ones below: > > 14/10/09 19:37:19 ERROR remote.EndpointWriter: AssociationError > [akka.tcp://sparkwor...@spark-slave1.domain.org:7078] -> [akka.tcp:// > sparkexecu...@spark-slave1.domain.org:53757]: Error [Association failed > with [akka.tcp://sparkexecu...@spark-slave1.domain.org:53757]] [ > akka.remote.EndpointAssociationException: Association failed with > [akka.tcp://sparkexecu...@spark-slave1.domain.org:53757] > Caused by: > akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2: > Connection refused: spark-slave1.domain.org/10.0.6.228:53757 > ] > 14/10/09 19:37:19 ERROR remote.EndpointWriter: AssociationError > [akka.tcp://sparkwor...@spark-slave1.domain.org:7078] -> [akka.tcp:// > sparkexecu...@spark-slave1.domain.org:53757]: Error [Association failed > with [akka.tcp://sparkexecu...@spark-slave1.domain.org:53757]] [ > akka.remote.EndpointAssociationException: Association failed with > [akka.tcp://sparkexecu...@spark-slave1.domain.org:53757] > Caused by: > akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2: > Connection refused: spark-slave1.domain.org/10.0.6.228:53757 > ] > > I'm a little confused because I can't find any solution to my problem. > I use Cloudera hadoop with spark. > > Best regards, > > Morbious >
