[jira] [Commented] (YARN-4162) Scheduler info in REST, is currently not displaying partition specific queue information similar to UI

[Wangda Tan (JIRA)]

    [ 
https://issues.apache.org/jira/browse/YARN-4162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936444#comment-14936444
 ] 

Wangda Tan commented on YARN-4162:
----------------------------------

Hi [~Naganarasimha],

Thanks for working on this, I have some comments about API design:

I found you added a new interface: {{@Path("/scheduler/partitions")}}.

I'm wondering instead of doing this, is it possible to change existing REST 
API? For example: {{ws/v1/cluster/scheduler}}. We have two major places need to 
change:
- Capacity, all different capacity-by-partitions.
- Resource, all resources-by-partitions.

The API in my mind looks like:
{code}
"queue": [
  {
    < Existing fields >
    "-xsi:type": "capacitySchedulerLeafQueueInfo",
    "capacity": "50.0",
    "usedCapacity": "0.0",
    "maxCapacity": "100.0",
    "absoluteCapacity": "50.0",
    "absoluteMaxCapacity": "100.0",
    "absoluteUsedCapacity": "0.0",
    "numApplications": "2",
    "queueName": "a",
    "state": "RUNNING",
    "resourcesUsed": {
      "memory": "0",
      "vCores": "0"
    },

    -- New Added Fields --
    "capacities": {
        "DEFAULT_PARTITION": {
            ....
        }
        "label-x" : {
            "capacity": 50.0,
            "usedCapacity": 40.0,
            "maxCapacity": ...
        }
    },

    "resources": {
        "DEFAULT_PARTITION": {
            "used": {
                "memory": 0,
                "vCores": 0
            },
            "reserved": {
                "memory": 0,
                "vCores": 0
            }
        },
        "label-x": {
            ....
        }
    }
  }
]
{code}

And also for user:
{code}
    ...
    "users": {
      "user": {
        "username": "wtan",
        "resourcesUsed": {
          "memory": "0",
          "vCores": "0"
        },

        -- New Added Fields --
        "resources": {
            "DEFAULT_PARTITION": {
                "used": {
                    "memory": 0,
                    "vCores": 0
                },
                "reserved": {
                    "memory": 0,
                    "vCores": 0
                }
            },
            "label-x": {
                ....
            }
        }
      }
    }
{code}

I think we may need to consider add something like QueueCapacitiesInfo and 
ResourceUsageInfo, which could be converted from a give 
QueueCapacities/ResourceUsage.

Thoughts?

> Scheduler info in REST, is currently not displaying partition specific queue 
> information similar to UI
> ------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-4162
>                 URL: https://issues.apache.org/jira/browse/YARN-4162
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: api, client, resourcemanager
>            Reporter: Naganarasimha G R
>            Assignee: Naganarasimha G R
>         Attachments: YARN-4162.v1.001.patch
>
>
> When Node Labels are enabled then REST Scheduler Information should also 
> provide partition specific queue information similar to the existing Web UI



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)