[
https://issues.apache.org/jira/browse/YARN-6789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16121998#comment-16121998
]
Wangda Tan commented on YARN-6789:
----------------------------------
[[email protected]], thanks for sharing your thoughts. Please see my
response below:
bq. If I understand correctly, what you're proposing is making the units a
global dictate. The RM says that memory has the unit "Mi", therefore all NMs
and clients must use that unit.
This is true, unit of values for different resource types are fixed. To be more
clear, I propose to remove the "unit" from ResourceInformation object. "Unit"
for each resource types are unwritten rules which each daemon/clients/am should
know that. "Unit" can be set for ResourceTypeInformation (by setting
resource-types.xml), however it is more like documentation instead of affecting
internal logics.
bq. What happens if an admin changes the units for a resource type?
This is not allowed in my proposal, I doubt that in what scenario we need to
change global units. For example, MB as unit for memory works so well today, I
don't think it make sense to upgrade it to GB/TB since value is a 64 bits
integer.
This is majorly to avoid dealing with backward compatibility and inconsistency
between daemons and clients. As I stated above, what if an old client talk to
RM asks 4096 memory without specifying "MB" as unit? If we make it to be
globally fixed and unwritten rule, we don't need to worry about inconsistency
any more. And all rolling upgrade problems are gone for the unit.
Thoughts?
> new api to get all supported resources from RM
> ----------------------------------------------
>
> Key: YARN-6789
> URL: https://issues.apache.org/jira/browse/YARN-6789
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
> Reporter: Sunil G
> Assignee: Sunil G
> Attachments: YARN-6789-YARN-3926.001.patch
>
>
> It will be better to provide an api to get all supported resource types from
> RM.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
