Hi, My requirement is to run Spark on Yarn without using the script spark-submit.
I have a servlet and a tomcat server. As and when request comes, it creates
a new SC and keeps it alive for the further requests, I ma setting my
master in sparkConf
as sparkConf.setMaster("yarn-cluster")
but the request is stuck indefinitely.
This works when I set
sparkConf.setMaster("yarn-client")
I am not sure, why is it not launching job in yarn-cluster mode.
Any thoughts?
Thanks and Regards,
Archit Thakur.
