Hi Steve, Your spark master is not running if you have not started it. On windows its missing come scripts/and or the correct installation instructions. I was able to start the master with
C:\> spark-class.cmd org.apache.spark.deploy.master.Master Then on the browser with localhost:port you get the “spark url”, with the 7707 port. This spark url has to be used to start the Spark worker C:\>spark-class.cmd org.apache.spark.deploy.worker.Worker spark://D-113052037.wipro.com:7077 Also I ran the master and worker on different windows machines. Also I don’t think the spark-shell is an alternative to starting the spark master. The Spark instructions and most of the scripts are geared towards unix systems. Regards, Prajod From: Steve Lewis [mailto:lordjoe2...@gmail.com] Sent: Thursday, August 21, 2014 10:08 PM To: user@spark.apache.org Subject: I am struggling to run Spark Examples on my local machine I download the binaries for spark-1.0.2-hadoop1 and unpack it on my Widows 8 box. I can execute spark-shell.com<http://spark-shell.com> and get a command window which does the proper things I open a browser to http:/localhost:4040 and a window comes up describing the spark-master Then using IntelliJ I create a project with JavaWordCount from the spark distribution. add When I run the job with the -Dspark.master=spark://local[*]:7707 (I have tried MANY other string) the Job fails for failure to connect to the spark master. So my question is 1) Do I have a spark-master running? How can I tell? doesn't the web page say it is running 2) How to I find the port on which the master is running and test that it is accepting jobs 3) Are there other steps I need to take before I can run a simple spark sample? 14/08/21 09:27:08 INFO scheduler.TaskSchedulerImpl: Adding task set 1.0 with 1 tasks 14/08/21 09:27:23 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory ... 14/08/21 09:28:08 ERROR cluster.SparkDeploySchedulerBackend: Application has been killed. Reason: All masters are unresponsive! Giving up. 14/08/21 09:28:08 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 1.0, whose tasks have all completed, from pool 14/08/21 09:28:08 INFO scheduler.TaskSchedulerImpl: Cancelling stage 1 14/08/21 09:28:08 INFO scheduler.DAGScheduler: Failed to run collect at JavaWordCount.java:68 Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: All masters are unresponsive! Giving up. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com
