GitHub user thide opened a pull request:
https://github.com/apache/spark/pull/18846
[SPARK-21642][CORE] Use FQDN for DRIVER_HOST_ADDRESS instead of ip address
## What changes were proposed in this pull request?
The patch lets spark web ui use FQDN as its hostname
Github user thide commented on the issue:
https://github.com/apache/spark/pull/18846
It seems that Spark started using IP address in SPARK-6440
(https://github.com/apache/spark/pull/5424). As far as I looked at the change,
this change was introduced only for visibility.
---
If
Github user thide commented on the issue:
https://github.com/apache/spark/pull/18846
Sorry, I forgot to answer one of your questions.
> Did you test if the host doesn't have fqdn it falls back to ip?
Yes, I tested it manually. InetAddress.getCanonical
Github user thide commented on the issue:
https://github.com/apache/spark/pull/18846
In my understanding, this does not affect driver/executor's heartbeat
behavior since this PR only changes DRIVER_HOST_ADDRESS.
However, I am not sure why the unit test was f
Github user thide commented on the issue:
https://github.com/apache/spark/pull/18846
@zsxwing Thank you for the pointer. I tested manually, as far as I tested,
Spark works as expected even if we apply this patch. I was able to confirm that
driver/executor shut down when its
