On Fri, Dec 9, 2016 at 9:42 PM, Chawla,Sumit <[email protected]> wrote:
> Hi All > > I am a new member of this group. Sorry if this question has already been > asked. I am looking for some information regarding Spark Mesos Integration: > > 1. How does Mesos schedules and launch Spark Executors? (Any pointer to > code will be helpful) > This is the scheduler: https://github.com/apache/spark/blob/master/resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala#L48 > > 2. How does Mesos frontend the Spark Administration capabilities? Are > there any urls through which mesos proxy's the request for spark master. I > am particularly interested to find out how to access Spark Master Admin and > Monitoring URLs when running using Mesos. > The driver's UI can be viewed on the node running the driver, by default on port 4040. Since this dies when the driver dies, you probably want to set up the history server http://spark.apache.org/docs/latest/monitoring.html > > 3. Any other good documentations/architecture for me to get started on > Mesos and Spark integration and internals. > Just the docs, and maybe read through the JIRAs and old emails. > > > Regards > Sumit Chawla > > -- Michael Gummelt Software Engineer Mesosphere
