Re: Spark WebUI link problem in Mesos Master
Hi Tim, thanks for clarifying this! Regarding dispatcher and cluster mode, it seems, that there’s still no way to get to exact Spark executor UI but only driver configuration and execution stats. -- Anton Kirillov Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Wednesday, July 29, 2015 at 7:48 PM, Tim Chen wrote: Hi Anton, Client mode we haven't populated the webui link and only did so for cluster mode. If you like you can open a JIRA and it should be a easy ticket for anyone to work on. Tim On Wed, Jul 29, 2015 at 4:27 AM, Anton Kirillov antonv.kiril...@gmail.com (mailto:antonv.kiril...@gmail.com) wrote: Hi everyone, I’m trying to get access to Spark web UI from Mesos Master but with no success: the host name displayed properly, but the link is not active, just text. Maybe it’s a well-known issue or I misconfigured something, but this problem is really annoying. Jobs are executed in client mode, framework registers successfully and execution happens as expected. Spark UI is available on client node but in Mesos Master’s interface hostname is just text. When launching in cluster mode (dispatcher is already launched) there’s only drivers information available with no reference to driver UI. I’m using Mesos 0.23.0, Spark 1.4.1 with binary executor (I’m using precompiled one) deployed to S3. Here’s the contents of spark-env.sh (http://spark-env.sh): export MESOS_NATIVE_JAVA_LIBRARY=/usr/lib/libmesos.so export SPARK_EXECUTOR_URI=https://S3 Path/spark-1.4.1-bin-hadoop2.6.tgz export MASTER=mesos://zk://zookeeper hosts/mesos export SPARK_PUBLIC_DNS=client machine EC2 public DNS name Maybe I miss something, but really appreciate any help! -- Anton Kirillov Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
Re: Spark WebUI link problem in Mesos Master
Hi Anton, Client mode we haven't populated the webui link and only did so for cluster mode. If you like you can open a JIRA and it should be a easy ticket for anyone to work on. Tim On Wed, Jul 29, 2015 at 4:27 AM, Anton Kirillov antonv.kiril...@gmail.com wrote: Hi everyone, I’m trying to get access to Spark web UI from Mesos Master but with no success: the host name displayed properly, but the link is not active, just text. Maybe it’s a well-known issue or I misconfigured something, but this problem is really annoying. Jobs are executed in client mode, framework registers successfully and execution happens as expected. Spark UI is available on client node but in Mesos Master’s interface hostname is just text. When launching in cluster mode (dispatcher is already launched) there’s only drivers information available with no reference to driver UI. I’m using Mesos 0.23.0, Spark 1.4.1 with binary executor (I’m using precompiled one) deployed to S3. Here’s the contents of spark-env.sh: export MESOS_NATIVE_JAVA_LIBRARY=/usr/lib/libmesos.so export SPARK_EXECUTOR_URI=https://S3 Path/spark-1.4.1-bin-hadoop2.6.tgz export MASTER=mesos://zk://zookeeper hosts/mesos export SPARK_PUBLIC_DNS=client machine EC2 public DNS name Maybe I miss something, but really appreciate any help! -- Anton Kirillov Sent with Sparrow http://www.sparrowmailapp.com/?sig
Spark WebUI link problem in Mesos Master
Hi everyone, I’m trying to get access to Spark web UI from Mesos Master but with no success: the host name displayed properly, but the link is not active, just text. Maybe it’s a well-known issue or I misconfigured something, but this problem is really annoying. Jobs are executed in client mode, framework registers successfully and execution happens as expected. Spark UI is available on client node but in Mesos Master’s interface hostname is just text. When launching in cluster mode (dispatcher is already launched) there’s only drivers information available with no reference to driver UI. I’m using Mesos 0.23.0, Spark 1.4.1 with binary executor (I’m using precompiled one) deployed to S3. Here’s the contents of spark-env.sh: export MESOS_NATIVE_JAVA_LIBRARY=/usr/lib/libmesos.so export SPARK_EXECUTOR_URI=https://S3 Path/spark-1.4.1-bin-hadoop2.6.tgz export MASTER=mesos://zk://zookeeper hosts/mesos export SPARK_PUBLIC_DNS=client machine EC2 public DNS name Maybe I miss something, but really appreciate any help! -- Anton Kirillov Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
