[
https://issues.apache.org/jira/browse/SPARK-2100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
DB Tsai updated SPARK-2100:
---------------------------
Description:
Since we want to use Spark hadoop APIs in local mode for design time to explore
the first couple hundred lines of data in HDFS. Also, we want to use Spark in
our tomcat application, so starting a jetty UI will make our tomcat unhappy. In
those scenarios, Spark UI is not necessary, and wasting resource.
As a result, for local mode, it's desirable that users are able to disable the
spark UI.
Couple places I found where the jetty will be started.
In SparkEnv.scala
1) val broadcastManager = new BroadcastManager(isDriver, conf, securityManager)
2) val httpFileServer = new HttpFileServer(securityManager)
httpFileServer.initialize()
I don't know if broadcastManager is needed in local mode tho.
was:
Since we're using Spark hadoop APIs in local mode for design time to explore
the first couple hundred lines of data in HDFS. Also, we use it in our tomcat
application, and starting a jetty UI will make our tomcat unhappy. In those
scenarios, Spark UI is not necessary, and wasting resource.
As a result, for local mode, users should be able to disable the spark UI.
Couple places I found that the jetty will be started.
In SparkEnv.scala
1) val broadcastManager = new BroadcastManager(isDriver, conf, securityManager)
2) val httpFileServer = new HttpFileServer(securityManager)
httpFileServer.initialize()
I don't know if broadcastManager is needed in local mode tho.
> Allow users to disable Jetty Spark UI in local mode
> ---------------------------------------------------
>
> Key: SPARK-2100
> URL: https://issues.apache.org/jira/browse/SPARK-2100
> Project: Spark
> Issue Type: Improvement
> Reporter: DB Tsai
>
> Since we want to use Spark hadoop APIs in local mode for design time to
> explore the first couple hundred lines of data in HDFS. Also, we want to use
> Spark in our tomcat application, so starting a jetty UI will make our tomcat
> unhappy. In those scenarios, Spark UI is not necessary, and wasting resource.
> As a result, for local mode, it's desirable that users are able to disable
> the spark UI.
> Couple places I found where the jetty will be started.
> In SparkEnv.scala
> 1) val broadcastManager = new BroadcastManager(isDriver, conf,
> securityManager)
> 2) val httpFileServer = new HttpFileServer(securityManager)
> httpFileServer.initialize()
> I don't know if broadcastManager is needed in local mode tho.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
