[
https://issues.apache.org/jira/browse/SPARK-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-6135.
------------------------------
Resolution: Not a Problem
We use pull requests rather than patches to propose a change. That said, this
is definitely not a valid change. This is not an exhaustive list of TLDs, nor
is it true that host names in this context even have a TLD.
> No checks of illegal hostname when runing spark on yarn.
> --------------------------------------------------------
>
> Key: SPARK-6135
> URL: https://issues.apache.org/jira/browse/SPARK-6135
> Project: Spark
> Issue Type: Bug
> Components: YARN
> Affects Versions: 1.2.1
> Environment: spark on yarn cluster.
> Reporter: Xia Hu
> Labels: spark-submit, yarn-client
> Attachments: check_hostname.patch
>
>
> I submit spark application to yarn cluster from a spark client machine, then
> I find this problem. When using spark on yarn-client model, driver is running
> in client, then the yarn cluster need to remotely connect to driver. But
> spark use InetAddress to read out the hostname of client machine,without
> checking if the hostname is legal or useful. So in my condition, some client
> machine have hostnames like "sjs_1_2", then this application fail because of
> cannot connect to driver on "sjs_1_2".
> I suppose there should be a check for if a hostname is legal, and if not,
> using the IP instead.
> And for this problem, I found an env. configuration "SPARK_LOCAL_HOSTNAME"
> can be used. if I set "SPARK_LOCAL_HOSTNAME" to be IP address in
> spark-env.sh, then this problem is solved. But it seems this configuration
> isn't introduced in any introductions or references and I found it when
> reading codes.
> But I still think a check of if hostname is illegal is needed,
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
