Canbin Zheng created FLINK-16602:
------------------------------------
Summary: Rework the Service design for Kubernetes deployment
Key: FLINK-16602
URL: https://issues.apache.org/jira/browse/FLINK-16602
Project: Flink
Issue Type: Improvement
Components: Deployment / Kubernetes
Affects Versions: 1.10.0
Reporter: Canbin Zheng
Fix For: 1.11.0
{color:#0e101a}At the moment we usually create two Services for a Flink
application, one is the internal Service and the other is the so-called rest
Service, the previous aims for forwarding request from the TMs to the JM, and
the rest Service mainly serves as an external service for the Flink
application. Here is a summary of the issues:{color}
# {color:#0e101a}The functionality boundary of the two Services is not clear
enough since the internal Service could also become the rest Service when its
exposed type is ClusterIP.{color}
# {color:#0e101a}For the high availability scenario, we create a useless
internal Service which does not help forward the internal requests since the
TMs directly communicate with the JM via the IP or hostname of the JM
Pod.{color}
# {color:#0e101a}Headless service is enough to help forward the internal
requests from the TMs to the JM. Service of ClusterIP type would add
corresponding rules into the iptables, too many rules in the iptables would
lower the kube-proxy's efficiency in refreshing iptables while notified of
change events, which could cause severe stability problems in a Kubernetes
cluster.{color}
{color:#0e101a}Therefore, we propose some improvements to the current
design:{color}
# {color:#0e101a}Clarify the functionality boundary for the two Services, the
internal Service only serves the internal communication from TMs to JM, while
the rest Service makes the Flink cluster accessible from outside. The internal
Service only exposes the RPC and BLOB ports while the external one exposes the
REST port.{color}
# {color:#0e101a}Do not create the internal Service in the high availability
case.{color}
# {color:#0e101a}Use HEADLESS type for the internal Service.{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
