[
https://issues.apache.org/jira/browse/FLINK-25764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Niklas Semmler updated FLINK-25764:
-----------------------------------
Description:
In the [docker
entrypoint|https://github.com/apache/flink-docker/blob/master/1.14/scala_2.12-java8-debian/docker-entrypoint.sh],
the JOB_MANAGER_RPC_ADDRESS is set to the current host by default (line 25).
This environment variable overrides the value set for jobmanager.rpc.address in
the flink config (line 78, 71). For the TaskManager, this means that it tries
to find the JobManager on the same host. When this is not the case, the
TaskManager will retry and ultimately terminate. Hence, for cluster
deployments, the variable has to be defined when starting the docker.
For Kubernetes deployments, the TaskManager cannot connect to the
jobmanager.rpc.address even when it is defined by the flink configmap. However,
we don't see this problem pop up, because for now the configmap is mounted
read-only into the containers (see FLINK-21383 for more details).
To simplify this configuration, I propose to (a) never set a default setting
for JOB_MANAGER_RPC_ADDRESS at all or (b) never set a default setting for any
non-JobManager container. The only down-side is that all docker deployments
will have to define JOB_MANAGER_RPC_ADDRESS, even when TaskManager and
JobManager run on the same node.
was:
In the [docker
entrypoint|https://github.com/apache/flink-docker/blob/master/1.14/scala_2.12-java8-debian/docker-entrypoint.sh],
the JOB_MANAGER_RPC_ADDRESS is set to the current host by default (line 25).
This environment variable overrides the value set for jobmanager.rpc.address in
the flink config (line 78, 71). For the TaskManager, this means that it tries
to find the JobManager on the same host. When this is not the case, the
TaskManager will retry and ultimately terminate. Hence, for cluster
deployments, the variable has to be defined when starting the docker.
For Kubernetes deployments, the TaskManager cannot connect to the
jobmanager.rpc.address even when it is defined by the flink configmap. However,
we don't see this problem pop up, because for now the configmap is mounted
read-only into the containers (see FLINK-21383 for more details).
To simplify this configuration, I propose to (a) never set a default setting
for JOB_MANAGER_RPC_ADDRESS at all or (b) never set a default setting for any
non-JobManager container. The only down-side is that all docker deployments
will have to define JOB_MANAGER_RPC_ADDRESS, even when TaskManager and
JobManager run on the same node.
Docker entrypoint:
https://github.com/apache/flink-docker/blob/master/1.14/scala_2.12-java8-debian/docker-entrypoint.sh
> Docker sets JobManager's rpc address to same host by default
> ------------------------------------------------------------
>
> Key: FLINK-25764
> URL: https://issues.apache.org/jira/browse/FLINK-25764
> Project: Flink
> Issue Type: Bug
> Components: Deployment / Kubernetes, flink-docker
> Affects Versions: 1.15.0
> Reporter: Niklas Semmler
> Priority: Major
> Labels: usability
>
> In the [docker
> entrypoint|https://github.com/apache/flink-docker/blob/master/1.14/scala_2.12-java8-debian/docker-entrypoint.sh],
> the JOB_MANAGER_RPC_ADDRESS is set to the current host by default (line 25).
> This environment variable overrides the value set for jobmanager.rpc.address
> in the flink config (line 78, 71). For the TaskManager, this means that it
> tries to find the JobManager on the same host. When this is not the case, the
> TaskManager will retry and ultimately terminate. Hence, for cluster
> deployments, the variable has to be defined when starting the docker.
> For Kubernetes deployments, the TaskManager cannot connect to the
> jobmanager.rpc.address even when it is defined by the flink configmap.
> However, we don't see this problem pop up, because for now the configmap is
> mounted read-only into the containers (see FLINK-21383 for more details).
> To simplify this configuration, I propose to (a) never set a default setting
> for JOB_MANAGER_RPC_ADDRESS at all or (b) never set a default setting for any
> non-JobManager container. The only down-side is that all docker deployments
> will have to define JOB_MANAGER_RPC_ADDRESS, even when TaskManager and
> JobManager run on the same node.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
