[
https://issues.apache.org/jira/browse/SPARK-15829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325001#comment-15325001
]
Andrew Davidson commented on SPARK-15829:
-----------------------------------------
Hi Xin
I ran netstat on my master. I do not think the port are in use.
To submit in cluster mode I use port 6066. If you are using port 7077 you are
in client mode. In client mode the application UI will run on the spark master.
In cluster mode the application UI runs on which ever slave the driver is
running on. If you notice in my original description the url is incorrect. the
ip is wrong, the port is correct.
Kind regards
Andy
# bash-4.2# netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address
State PID/Program name
tcp 0 0 0.0.0.0:8652 0.0.0.0:*
LISTEN 3832/gmetad
tcp 0 0 0.0.0.0:8787 0.0.0.0:*
LISTEN 2584/rserver
tcp 0 0 0.0.0.0:36757 0.0.0.0:*
LISTEN 2905/java
tcp 0 0 0.0.0.0:50070 0.0.0.0:*
LISTEN 2905/java
tcp 0 0 0.0.0.0:22 0.0.0.0:*
LISTEN 2144/sshd
tcp 0 0 127.0.0.1:631 0.0.0.0:*
LISTEN 2095/cupsd
tcp 0 0 127.0.0.1:7000 0.0.0.0:*
LISTEN 6512/python3.4
tcp 0 0 127.0.0.1:25 0.0.0.0:*
LISTEN 2183/sendmail
tcp 0 0 0.0.0.0:43813 0.0.0.0:*
LISTEN 3093/java
tcp 0 0 172.31.22.140:9000 0.0.0.0:*
LISTEN 2905/java
tcp 0 0 0.0.0.0:8649 0.0.0.0:*
LISTEN 3810/gmond
tcp 0 0 0.0.0.0:50090 0.0.0.0:*
LISTEN 3093/java
tcp 0 0 0.0.0.0:8651 0.0.0.0:*
LISTEN 3832/gmetad
tcp 0 0 :::8080 :::*
LISTEN 23719/java
tcp 0 0 :::8081 :::*
LISTEN 5588/java
tcp 0 0 ::ffff:172.31.22.140:6066 :::*
LISTEN 23719/java
tcp 0 0 ::ffff:172.31.22.140:6067 :::*
LISTEN 5588/java
tcp 0 0 :::22 :::*
LISTEN 2144/sshd
tcp 0 0 ::1:631 :::*
LISTEN 2095/cupsd
tcp 0 0 :::19998 :::*
LISTEN 3709/java
tcp 0 0 :::19999 :::*
LISTEN 3709/java
tcp 0 0 ::ffff:172.31.22.140:7077 :::*
LISTEN 23719/java
tcp 0 0 ::ffff:172.31.22.140:7078 :::*
LISTEN 5588/java
udp 0 0 0.0.0.0:8649 0.0.0.0:*
3810/gmond
udp 0 0 0.0.0.0:631 0.0.0.0:*
2095/cupsd
udp 0 0 0.0.0.0:38546 0.0.0.0:*
2905/java
udp 0 0 0.0.0.0:68 0.0.0.0:*
1142/dhclient
udp 0 0 172.31.22.140:123 0.0.0.0:*
2168/ntpd
udp 0 0 127.0.0.1:123 0.0.0.0:*
2168/ntpd
udp 0 0 0.0.0.0:123 0.0.0.0:*
2168/ntpd
bash-4.2#
> spark master webpage links to application UI broke when running in cluster
> mode
> -------------------------------------------------------------------------------
>
> Key: SPARK-15829
> URL: https://issues.apache.org/jira/browse/SPARK-15829
> Project: Spark
> Issue Type: Bug
> Components: EC2
> Affects Versions: 1.6.1
> Environment: AWS ec2 cluster
> Reporter: Andrew Davidson
> Priority: Critical
>
> Hi
> I created a cluster using the spark-1.6.1-bin-hadoop2.6/ec2/spark-ec2
> I use the stand alone cluster manager. I have a streaming app running in
> cluster mode. I notice the master webpage links to the application UI page
> are incorrect
> It does not look like jira will let my upload images. I'll try and describe
> the web pages and the bug
> My master is running on
> http://ec2-54-215-230-73.us-west-1.compute.amazonaws.com:8080/
> It has a section marked "applications". If I click on one of the running
> application ids I am taken to a page showing "Executor Summary". This page
> has a link to teh 'application detail UI' the url is
> http://ec2-54-215-230-73.us-west-1.compute.amazonaws.com:4041/
> Notice it things the application UI is running on the cluster master.
> It is actually running on the same machine as the driver on port 4041. I was
> able to reverse engine the url by noticing the private ip address is part of
> the worker id . For example worker-20160322041632-172.31.23.201-34909
> next I went on the aws ec2 console to find the public DNS name for this
> machine
> http://ec2-54-193-104-169.us-west-1.compute.amazonaws.com:4041/streaming/
> Kind regards
> Andy
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
