Varun Vasudev updated YARN-4081:
    Attachment: YARN-4081-YARN-3926.003.patch

1) Uses String instead of URI for resource key? I think it maybe more efficient 
to use String, it will be easier to construct when using it, uses less resource 
(hasn't tested, but I think it will be true according to #fields in String and 
URI). I can understand the motivation of solving conflicts of resource 
namespace, but I think namespace conflict is not the major use case AND String 
can define namespace as well.


2) Relationship between ResourceInformation and ResourceMapEntry: currently 
it's 1-1 mapping, a ResourceInformation has value/unit from ResourceMapEntry, 
they're kind of overlapping and also confusing. I think it's better to make 
ResourceInformation to be one for each resource type. ResourceMapEntry contains 
runtime information, and ResourceInformation contains configured information. 
This will also avoid create ResourceInformation instance when invoking 

3) Resource unit: I like the design which can easily convert a internal value 
to human-readable value. But I think maybe we don't need to support define unit 
in ResourceMapEntry. There're some cons of it:

    When we doing comparision of resources, we have to convert units, it's an 
extra overhead.
    It doesn't make a lot of sense to me that keep internal unit of resources: 
We should handle it when constructing Resource (something like 
Resource.newInstance("memory", 12, "GB")). And we will use the standard unit to 
do internal computations.
    We can define the standard unit in each "ResourceInformation" if you agree 
with #2.

I spoke with Wangda offline an we agree that it makes more sense to do 
performance testing once we have the DRC changes in. Since this patch is going 
in to a branch, there's no issue committing and running a full suite of 
performance tests once we have the DRC changes in.

4) Do you think it's better to have a global ResourceInformation map instead of 
storing it in each Resource instance?

For now, I'd like to keep it per-resource instance but if it becomes an 
overhead, we can make it a global instance.

5) Resource#compareTo/hashCode has debug logging.


6) It seems not necessary to instance ArrayList in Resource#compareTo. Just 
traverse the set can avoid create the temporary ArrayList.
Good point. I decided to use the size of the set itself as the sort order and 
avoid the issue altogether.

> Add support for multiple resource types in the Resource class
> -------------------------------------------------------------
>                 Key: YARN-4081
>                 URL: https://issues.apache.org/jira/browse/YARN-4081
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Varun Vasudev
>            Assignee: Varun Vasudev
>         Attachments: YARN-4081-YARN-3926.001.patch, 
> YARN-4081-YARN-3926.002.patch, YARN-4081-YARN-3926.003.patch
> For adding support for multiple resource types, we need to add support for 
> this in the Resource class.

This message was sent by Atlassian JIRA

Reply via email to