[
https://issues.apache.org/jira/browse/SPARK-13041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stavros Kontopoulos updated SPARK-13041:
----------------------------------------
Description:
It would be convenient to have the uri of the driver's ui and the driver's
mesos sandbox uri on the dispatcher's ui. Things to consider:
a) sandbox link has the master's hostname name in it which may not be
accessible if zk is used.
b) In case we have multiple executors on the same machine we make sure the
dispatcher allocates a known port from the available offered, so the port value
is known upfront to present it.
c) we need to make sure there are no conflicts with port isolation on slaves,
eg multiple drivers or multiple executors. Right now the Mesos containerizer
doesn't provide much isolation:
http://mesos.apache.org/documentation/latest/containerizer/, for port isolation
you need: http://mesos.apache.org/documentation/latest/network-monitoring/
was:
It would be convenient to have the uri of the driver's ui and the driver's
mesos sandbox uri on the dispatcher's ui. Two things to consider:
a) sandbox link has the master's hostname name in it which may not be
accessible if zk is used.
b) In case we have multiple executors on the same machine we make sure the
dispatcher allocates a known port from the available offered, so the port value
is known upfront to present it.
c) we need to make sure there are no conflicts with port isolation on slaves,
eg multiple drivers or multiple executors. Right now the Mesos containerizer
doesn't provide much isolation:
http://mesos.apache.org/documentation/latest/containerizer/, for port isolation
you need: http://mesos.apache.org/documentation/latest/network-monitoring/
> Add a driver ui link and a mesos sandbox link on the dispatcher's ui page for
> each driver
> -----------------------------------------------------------------------------------------
>
> Key: SPARK-13041
> URL: https://issues.apache.org/jira/browse/SPARK-13041
> Project: Spark
> Issue Type: Improvement
> Components: Mesos
> Reporter: Stavros Kontopoulos
> Priority: Minor
>
> It would be convenient to have the uri of the driver's ui and the driver's
> mesos sandbox uri on the dispatcher's ui. Things to consider:
> a) sandbox link has the master's hostname name in it which may not be
> accessible if zk is used.
> b) In case we have multiple executors on the same machine we make sure the
> dispatcher allocates a known port from the available offered, so the port
> value is known upfront to present it.
> c) we need to make sure there are no conflicts with port isolation on slaves,
> eg multiple drivers or multiple executors. Right now the Mesos containerizer
> doesn't provide much isolation:
> http://mesos.apache.org/documentation/latest/containerizer/, for port
> isolation you need:
> http://mesos.apache.org/documentation/latest/network-monitoring/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
