Spark UI port

2017-09-10 Thread Sunil Kalyanpur
Hello all, I am running PySpark Job (v2.0.2) with checkpoint enabled in Mesos cluster and am using Marathon for orchestration. When the job is restarted using Marathon, Spark UI is not getting started at the port specified by Marathon. Instead, it is picking port from the checkpoint. Is there a

Spark UI to use Marathon assigned port

2017-09-07 Thread Sunil Kalyanpur
Hello all, I am running PySpark Job (v2.0.2) with checkpoint enabled in Mesos cluster and am using Marathon for orchestration. When the job is restarted using Marathon, Spark UI is not getting started at the port specified by Marathon. Instead, it is picking port from the checkpoint. Is there a