Ruslan Dautkhanov created YARN-8981:
---------------------------------------
Summary: Virtual IP address support
Key: YARN-8981
URL: https://issues.apache.org/jira/browse/YARN-8981
Project: Hadoop YARN
Issue Type: New Feature
Affects Versions: 3.1.1, 3.0.2
Reporter: Ruslan Dautkhanov
I couldn't find support for virtual IP addresses in YARN framework.
This would be great if we have a docker-on-yarn service and if it for example
has to be failed over to another physical host, clients can still find it.
So the idea is for YARN to bring up that virtual IP address (an
additional/secondary IP address ) on a physical host where that particular
docker container is running, so the clients that use that container's services
don't have to change connection details every time that container moves around
in YARN cluster.
Similarly to virtual IP addresses in Kubernetes world:
[https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies]
One implementation could be through `ip address add` \ `ip address remove`.
Kubernetes uses a more complicated `kube-proxy`, similarly to `docker-proxy`
process in pure dockers / non-kubernetes docker deployments.
Another approach is running a separate DNS service for a DNS subdomain (main
DNS server would have to forward all requests for that DNS subdomain to a YARN
DNS service). In Oracle Clusterware similar process is called GNS:
https://docs.oracle.com/en/database/oracle/oracle-database/12.2/cwsol/about-the-grid-naming-service-vip-address.html#GUID-A4EE0CC6-A5F1-4507-82D6-D5C43E0F1584
Would be great to have support for either virtual IP addresses managed by YARN
directly or something similar to Oracle's GNS dns service.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
