[jira] [Comment Edited] (FLINK-13694) Refactor Flink on YARN configuration with relavent overlay classes
[
https://issues.apache.org/jira/browse/FLINK-13694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907089#comment-16907089
]
vinoyang edited comment on FLINK-13694 at 8/14/19 9:42 AM:
---
Our platform needs to provide a centralized submitter. We need to provide a
customized env variable {{HADOOP_USER_NAME}}, it is hardcoded in
{{ApplicationYarnClusterDescriptor}} :
{code:java}
appMasterEnv.put(YarnConfigKeys.ENV_HADOOP_USER_NAME,
UserGroupInformation.getCurrentUser().getUserName());
{code}
I prepared to give a config option. However, I found there is a better choice:
we can use {{HadoopUserOverlay}}. I found {{ContainerOverlay}} provides a good
abstract, but it only been used in Flink on Mesos, while it was defined in the
flink-runtime module.
IMO, we either use a unified abstract or move {{ContainerOverlay}} into Flink
on Mesos module.
was (Author: yanghua):
Our platform needs to provide a centralized submitter. We need to provide a
customized env variable {{HADOOP_USER_NAME}}, it is hardcoded in
{{ApplicationYarnClusterDescriptor}} :
{code:java}
appMasterEnv.put(YarnConfigKeys.ENV_HADOOP_USER_NAME,
UserGroupInformation.getCurrentUser().getUserName());
{code}
I prepared to give a config option. However, I found there is a better choice:
we can use {{HadoopUserOverlay}}. I found {{ContainerOverlay}} provides a good
abstract, but it only been used in Flink on Mesos, while it was defined in the
flink-runtime module.
IMO, we either use a unified abstract or move it into Flink on Mesos module.
> Refactor Flink on YARN configuration with relavent overlay classes
> --
>
> Key: FLINK-13694
> URL: https://issues.apache.org/jira/browse/FLINK-13694
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / YARN
>Reporter: vinoyang
>Priority: Major
>
> There is an abstract layer beyond special deployment named
> {{ContainerOverlay}}.
> However, currently, only Flink on Mesos used this mechanism. Flink on YARN
> still used some trivial configuration items in
> {{AbstractYarnClusterDescriptor}}. IMO we need to use container overlay to
> refactor Flink on YARN related implementation.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (FLINK-13694) Refactor Flink on YARN configuration with relavent overlay classes
[
https://issues.apache.org/jira/browse/FLINK-13694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907089#comment-16907089
]
vinoyang edited comment on FLINK-13694 at 8/14/19 9:41 AM:
---
Our platform needs to provide a centralized submitter. We need to provide a
customized env variable {{HADOOP_USER_NAME}}, it is hardcoded in
{{ApplicationYarnClusterDescriptor}} :
{code:java}
appMasterEnv.put(YarnConfigKeys.ENV_HADOOP_USER_NAME,
UserGroupInformation.getCurrentUser().getUserName());
{code}
I prepared to give a config option. However, I found there is a better choice:
we can use {{HadoopUserOverlay}}. I found {{ContainerOverlay}} provides a good
abstract, but it only been used in Flink on Mesos, while it was defined in the
flink-runtime module.
IMO, we either use a unified abstract or move it into Flink on Mesos module.
was (Author: yanghua):
We need to provide a customized env variable {{HADOOP_USER_NAME}}, it is
hardcoded in {{ApplicationYarnClusterDescriptor}} :
{code:java}
appMasterEnv.put(YarnConfigKeys.ENV_HADOOP_USER_NAME,
UserGroupInformation.getCurrentUser().getUserName());
{code}
I prepared to give a config option. However, I found there is a better choice:
we can use {{HadoopUserOverlay}}. I found {{ContainerOverlay}} provides a good
abstract, but it only been used in Flink on Mesos, while it was defined in the
flink-runtime module.
> Refactor Flink on YARN configuration with relavent overlay classes
> --
>
> Key: FLINK-13694
> URL: https://issues.apache.org/jira/browse/FLINK-13694
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / YARN
>Reporter: vinoyang
>Priority: Major
>
> There is an abstract layer beyond special deployment named
> {{ContainerOverlay}}.
> However, currently, only Flink on Mesos used this mechanism. Flink on YARN
> still used some trivial configuration items in
> {{AbstractYarnClusterDescriptor}}. IMO we need to use container overlay to
> refactor Flink on YARN related implementation.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
