[
https://issues.apache.org/jira/browse/YARN-10854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tao Yang updated YARN-10854:
----------------------------
Description:
Currently inactive nodes which have been decommissioned/shutdown/lost for a
while(specified expiration time defined via
{{yarn.resourcemanager.node-removal-untracked.timeout-ms}}, 60 seconds by
default) and not exist in both include and exclude files can be marked as
untracked nodes and can be removed from RM state (YARN-4311). It's very useful
when auto-scaling is enabled in elastic cloud environment, which can avoid
unlimited increase of inactive nodes (mostly are decommissioned nodes).
But this only works when the include path is configured, mismatched for most of
our cloud environments without configured white list of nodes, which can lead
to easily control for the auto-scaling of nodes without further security
requirements.
So I propose to support marking inactive node as untracked without configured
include path, to be compatible with the former versions, we can add a switch
config for this.
Any thoughts/suggestions/feedbacks are welcome!
was:
Currently inactive nodes which have been decommissioned/shutdown/lost for a
while(specified expiration time defined via
{{yarn.resourcemanager.node-removal-untracked.timeout-ms}}, 60 seconds by
default) and not exist in both include and exclude files can be marked as
untracked nodes and can be removed from RM state. It's very useful when
auto-scaling is enabled in elastic cloud environment, which can avoid unlimited
increase of inactive nodes (mostly are decommissioned nodes).
But this only works when the include path is configured, mismatched for most of
our cloud environments without configured white list of nodes, which can lead
to easily control for the auto-scaling of nodes without further security
requirements.
So I propose to support marking inactive node as untracked without configured
include path, to be compatible with the former versions, we can add a switch
config for this.
Any thoughts/suggestions/feedbacks are welcome!
> Support marking inactive node as untracked without configured include path
> --------------------------------------------------------------------------
>
> Key: YARN-10854
> URL: https://issues.apache.org/jira/browse/YARN-10854
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
> Reporter: Tao Yang
> Assignee: Tao Yang
> Priority: Major
> Attachments: YARN-10854.001.patch
>
>
> Currently inactive nodes which have been decommissioned/shutdown/lost for a
> while(specified expiration time defined via
> {{yarn.resourcemanager.node-removal-untracked.timeout-ms}}, 60 seconds by
> default) and not exist in both include and exclude files can be marked as
> untracked nodes and can be removed from RM state (YARN-4311). It's very
> useful when auto-scaling is enabled in elastic cloud environment, which can
> avoid unlimited increase of inactive nodes (mostly are decommissioned nodes).
> But this only works when the include path is configured, mismatched for most
> of our cloud environments without configured white list of nodes, which can
> lead to easily control for the auto-scaling of nodes without further security
> requirements.
> So I propose to support marking inactive node as untracked without configured
> include path, to be compatible with the former versions, we can add a switch
> config for this.
> Any thoughts/suggestions/feedbacks are welcome!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
