Are you running the driver program (that is your application process) in your desktop and trying to run it on the cluster in EC2? It could very well be a hostname mismatch in some way due to the all the public hostname, private hostname, private ip, firewall craziness of ec2. You have to probably try different combinations of hostname and ip. First of all, you could try to submit applications from with one of nodes inside ec2 to narrow down the problem. Additionally, see the Configuration page in the Spark Standalone documentation to configure which hostname is used (for binding) by Spark Standalone daemons.
TD On Fri, Mar 13, 2015 at 2:41 PM, Mohit Anchlia <mohitanch...@gmail.com> wrote: > I am running spark streaming standalone in ec2 and I am trying to run > wordcount example from my desktop. The program is unable to connect to the > master, in the logs I see, which seems to be an issue with hostname. > > 15/03/13 17:37:44 ERROR EndpointWriter: dropping message [class > akka.actor.ActorSelectionMessage] for non-local recipient [Actor[akka.tcp:// > sparkMaster@54.69.22.4:7077/]] arriving at [akka.tcp:// > sparkMaster@54.69.22.4:7077] inbound addresses are > [akka.tcp://sparkMaster@ip-10-241-251-232:7077] > >
