[
https://issues.apache.org/jira/browse/YUNIKORN-165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17503937#comment-17503937
]
Wilfred Spiegelenburg commented on YUNIKORN-165:
------------------------------------------------
Internally we must move to integers for the representation, just to fix the
issue described in YUNIKORN-1105. We should be able to handle about 9 exa byte
as the maximum size of a cluster using a standard int64 or 18 exa byte if we
use an unsigned integer. Not sure if we will overflow that in any resources
quickly.
What I meant with the staying closer to what K8s does is to allow specifying
the resources with a quantity suffix specifically in the configuration. This is
what we currently require:
{code:java}
resources:
max:
vcore: 21000
memory: 610000 {code}
That specifies 21 cpu's (base is milli vcores) and 610GB for memory (base is
1MB). Not that simple to interpret. We should allow 610G, and like with the
shim side we need to convert that to B. Same for the vcore, 21 should be
converted to 21000 m cores etc.
I would like to see us support a configuration like this:
{code:java}
resources:
guaranteed:
vcore: 2500m
memory: 600G
max:
vcore: 10
memory: 1T{code}
This is independent of how we represent the data in the REST specification.
REST should align with the base suffix: milli for vcores (base == 1e-3) and
bytes / undefined for others (base == 1). So never a quantity suffix in REST,
vcores always as milli vcores and all other resources as bytes or undefined.
> API response fields can be returned as proper data structure rather than free
> text
> ----------------------------------------------------------------------------------
>
> Key: YUNIKORN-165
> URL: https://issues.apache.org/jira/browse/YUNIKORN-165
> Project: Apache YuniKorn
> Issue Type: Improvement
> Components: core - common, webapp
> Reporter: Ayub Pathan
> Priority: Critical
>
> For most of the APIs, some of the fields are recorded as free text, this can
> be improved by sending them as proper data structure like map etc. This helps
> in easy consumption of the REST aPI response.
> For example:
> * Some fields from Apps API like resource,usedResource, partition can be
> returned as maps..
> {noformat}
> {
> "allocations": [
> {
> "allocationKey": "1e5225f7-4d60-400f-8b9b-5b7027b4c394",
> "allocationTags": null,
> "applicationId": "abcd",
> "nodeId": "docker-desktop",
> "partition": "default",
> "priority": "<nil>",
> "queueName": "root.development",
> "resource": "[memory:100 vcore:200]",
> "uuid": "b1b89eaf-81fb-4b2b-b20a-bf7bd96d3430"
> }
> ],
> "applicationID": "abcd",
> "applicationState": "Running",
> "partition": "[mycluster]default",
> "queueName": "root.development",
> "submissionTime": 1589740021170274400,
> "usedResource": "[memory:100 vcore:200]"
> }, {noformat}
>
> * same for nodes API response(fields like available, capacity, occupied,
> allocated) as well.
> {noformat}
> [
> {
> "nodesInfo": [
> {
> "RackName": "/rack-default",
> "allocated": "[memory:0 vcore:0]",
> "allocations": null,
> "available": "[ephemeral-storage:56450230179 hugepages-1Gi:0
> hugepages-2Mi:0 memory:1824 pods:110 vcore:3250]",
> "capacity": "[ephemeral-storage:56450230179 hugepages-1Gi:0
> hugepages-2Mi:0 memory:1982 pods:110 vcore:4000]",
> "hostName": "docker-desktop",
> "nodeID": "docker-desktop",
> "occupied": "[memory:158 vcore:750]",
> "schedulable": true
> }
> ],
> "partitionName": "[mycluster]default"
> }
> ] {noformat}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
