Wangda Tan commented on YARN-6789:

[~dan...@cloudera.com],  thanks for sharing your thoughts. Please see my 
response below:

bq. If I understand correctly, what you're proposing is making the units a 
global dictate. The RM says that memory has the unit "Mi", therefore all NMs 
and clients must use that unit. 
This is true, unit of values for different resource types are fixed. To be more 
clear, I propose to remove the "unit" from ResourceInformation object. "Unit" 
for each resource types are unwritten rules which each daemon/clients/am should 
know that. "Unit" can be set for ResourceTypeInformation (by setting 
resource-types.xml), however it is more like documentation instead of affecting 
internal logics.

bq. What happens if an admin changes the units for a resource type? 
This is not allowed in my proposal, I doubt that in what scenario we need to 
change global units. For example, MB as unit for memory works so well today, I 
don't think it make sense to upgrade it to GB/TB since value is a 64 bits 
This is majorly to avoid dealing with backward compatibility and inconsistency 
between daemons and clients. As I stated above, what if an old client talk to 
RM asks 4096 memory without specifying "MB" as unit? If we make it to be 
globally fixed and unwritten rule, we don't need to worry about inconsistency 
any more. And all rolling upgrade problems are gone for the unit. 


> new api to get all supported resources from RM
> ----------------------------------------------
>                 Key: YARN-6789
>                 URL: https://issues.apache.org/jira/browse/YARN-6789
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: YARN-6789-YARN-3926.001.patch
> It will be better to provide an api to get all supported resource types from 
> RM.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to