Vishwas Nalka created SPARK-29465:
-------------------------------------
Summary: Unable to configure SPARK UI (spark.ui.port) in spark
yarn cluster mode.
Key: SPARK-29465
URL: https://issues.apache.org/jira/browse/SPARK-29465
Project: Spark
Issue Type: Bug
Components: Spark Submit, YARN
Affects Versions: 2.4.4, 1.6.2
Reporter: Vishwas Nalka
I'm trying to restrict the ports used by spark app which is launched in yarn
cluster mode. All ports (viz. driver, executor, blockmanager) could be
specified using the respective properties except the ui port. The spark app is
launched using JAVA code and setting the property spark.ui.port in sparkConf
doesn't seem to help. Even setting a JVM option -Dspark.ui.port="some_port"
does not spawn the UI is required port.
>From the logs of the spark app, *_the property spark.ui.port is overridden and
>the JVM property '-Dspark.ui.port=0' is set_* even though it is never set to
>0.
_(Run in Spark 1.6.2) From the logs ->_
_command:LD_LIBRARY_PATH="/usr/hdp/2.6.4.0-91/hadoop/lib/native:$LD_LIBRARY_PATH"
\{{JAVA_HOME}}/bin/java -server -XX:OnOutOfMemoryError='kill %p' -Xms4096m
-Xmx4096m -Djava.io.tmpdir=\{{PWD}}/tmp '-Dspark.blockManager.port=9900'
'-Dspark.driver.port=9902' '-Dspark.fileserver.port=9903'
'-Dspark.broadcast.port=9904' '-Dspark.port.maxRetries=20' '-Dspark.ui.port=0'
'-Dspark.executor.port=9905'_
_19/10/14 16:39:59 INFO Utils: Successfully started service 'SparkUI' on port
35167.19/10/14 16:39:59 INFO SparkUI: Started SparkUI at_
[_http://10.65.170.98:35167_|http://10.65.170.98:35167/]
Even tried using a *spark-submit command with --conf spark.ui.port* does spawn
UI in required port
{color:#172b4d}_(Run in Spark 2.4.4)_{color}
{color:#172b4d}_./bin/spark-submit --class org.apache.spark.examples.SparkPi
--master yarn --deploy-mode cluster --driver-memory 4g --executor-memory 2g
--executor-cores 1 --conf spark.ui.port=12345 --conf spark.driver.port=12340
--queue default examples/jars/spark-examples_2.11-2.4.4.jar 10_{color}
_From the logs::_
_19/10/15 00:04:05 INFO ui.SparkUI: Stopped Spark web UI at
[http://invrh74ace005.informatica.com:46622|http://invrh74ace005.informatica.com:46622/]_
_command:\{{JAVA_HOME}}/bin/java -server -Xmx2048m
-Djava.io.tmpdir=\{{PWD}}/tmp '-Dspark.ui.port=0' 'Dspark.driver.port=12340'
-Dspark.yarn.app.container.log.dir=<LOG_DIR> -XX:OnOutOfMemoryError='kill %p'
org.apache.spark.executor.CoarseGrainedExecutorBackend --driver-url
spark://coarsegrainedschedu...@invrh74ace005.informatica.com:12340
--executor-id <executorId> --hostname <hostname> --cores 1 --app-id
application_1570992022035_0089 --user-class-path
file:$PWD/__app__.jar1><LOG_DIR>/stdout2><LOG_DIR>/stderr_
Looks like the application master override this and set a JVM property before
launch resulting in random UI port even though spark.ui.port is set.
In these links
#
[https://github.com/apache/spark/blob/master/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala]
(line 214)
#
[https://github.com/cloudera/spark/blob/master/yarn/alpha/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala]
(line 75)
I can see that the method _*run() in above files sets a system property
UI_PORT*_ and _*spark.ui.port respectively.*_
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
