[
https://issues.apache.org/jira/browse/SPARK-20578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-20578.
-------------------------------
Resolution: Invalid
Questions should go to the user mailing list. JIRA isn't for tech support.
> oozie submit spark job, the driver ui port is 0, instead of 4040
> ----------------------------------------------------------------
>
> Key: SPARK-20578
> URL: https://issues.apache.org/jira/browse/SPARK-20578
> Project: Spark
> Issue Type: Question
> Components: Web UI
> Affects Versions: 2.1.0
> Reporter: wuchang
>
> My spark Job is submmited by oozie in hue. The spark is running in
> yarn-cluster mode. I am trying to monitor the running application status by
> so-called driver's 4040 port, but I cannot find the 4040 port , I check the
> process:
> {code}
> appuser 137872 137870 0 18:55 ? 00:00:00 /bin/bash -c /home/jdk/bin/java
> -server -Xmx4096m
> -Djava.io.tmpdir=/data6/data/hadoop/tmp/usercache/appuser/appcache/application_1493800575189_0547/container_1493800575189_0547_01_000004/tmp
> '-Dspark.driver.port=36503' '-Dspark.ui.port=0'
> -Dspark.yarn.app.container.log.dir=/home/log/hadoop/logs/userlogs/application_1493800575189_0547/container_1493800575189_0547_01_000004
> -XX:OnOutOfMemoryError='kill %p'
> org.apache.spark.executor.CoarseGrainedExecutorBackend --driver-url
> spark://[email protected]:36503 --executor-id 3
> --hostname 10.120.117.100 --cores 1 --app-id application_1493800575189_0547
> --user-class-path
> file:/data6/data/hadoop/tmp/usercache/appuser/appcache/application_1493800575189_0547/container_1493800575189_0547_01_000004/__app__.jar
> 1>
> /home/log/hadoop/logs/userlogs/application_1493800575189_0547/container_1493800575189_0547_01_000004/stdout
> 2>
> /home/log/hadoop/logs/userlogs/application_1493800575189_0547/container_1493800575189_0547_01_000004/stderr
> appuser 138337 137872 99 18:55 ? 00:05:11 /home/jdk/bin/java -server
> -Xmx4096m
> -Djava.io.tmpdir=/data6/data/hadoop/tmp/usercache/appuser/appcache/application_1493800575189_0547/container_1493800575189_0547_01_000004/tmp
> -Dspark.driver.port=36503 -Dspark.ui.port=0
> -Dspark.yarn.app.container.log.dir=/home/log/hadoop/logs/userlogs/application_1493800575189_0547/container_1493800575189_0547_01_000004
> -XX:OnOutOfMemoryError=kill %p
> org.apache.spark.executor.CoarseGrainedExecutorBackend --driver-url
> spark://[email protected]:36503 --executor-id 3
> --hostname 10.120.117.100 --cores 1 --app-id application_1493800575189_0547
> --user-class-path
> file:/data6/data/hadoop/tmp/usercache/appuser/appcache/application_1493800575189_0547/container_1493800575189_0547_01_000004/__app__.jar
> {code}
> I don't know why the spark.ui.port is 0 instead of 4040. Of course , the port
> 0 is not allowed by my linux system.So , I cannot monitor the application
> status from REST api.
>
> Anybody can give me some suggestions?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
