[ 
https://issues.apache.org/jira/browse/MESOS-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Mahler reassigned MESOS-7883:
--------------------------------------

    Assignee: Benjamin Mahler
      Sprint: Resource Mgmt RI12 Sp 42

> Quota heuristic check not accounting for mount volumes
> ------------------------------------------------------
>
>                 Key: MESOS-7883
>                 URL: https://issues.apache.org/jira/browse/MESOS-7883
>             Project: Mesos
>          Issue Type: Bug
>          Components: allocation
>            Reporter: Vincent Roy
>            Assignee: Benjamin Mahler
>            Priority: Major
>              Labels: resource-management
>
> This may be expected but came as a surprise to us. We are unable to create a 
> quota bigger than the root disk space on slaves.
> Given two clusters with the same number of slaves and root disk size, but one 
> that also has mount volumes, is what the disk resources look like:
> {noformat}
> [root@fin-fang-foom-master-1 ~]# curl -s master.mesos:5050/state | jq 
> '.slaves[] .resources .disk'
> 28698
> 28699
> 28698
> 28698
> 28697
> {noformat}
> {noformat}
> [root@hydra-master-1 ~]# curl -s master.mesos:5050/state | jq '.slaves[] 
> .resources .disk'
> 50817
> 50817
> 50814
> 50819
> 50817
> {noformat}
> In {{fin-fang-foom}}, I was able to create a quota for {{143490mb}} which is 
> the total of available disk resources, root in this case, as reported by 
> Mesos. For {{hydra}}, I am only able to create a quota for {{143489mb}}. This 
> is equivalent to the total of root disks available in {{hydra}} rather than 
> the total available disks reported by Mesos resources which is {{254084mb}}.
> With a modified Mesos that adds logging to {{quota_handler}}, we can see that 
> only the {{disk(*)}} number increases in {{nonStaticClusterResources}} after 
> every iteration. The final iteration is {{disk(*):143489}} which is the 
> maximum quota I was able to create on {{hydra}}. We expected that quota 
> heuristic check would also include resources such as 
> {{disk(*)[MOUNT:/dcos/volume2]:7373}}
> {noformat}
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.763764 
> 24902 quota_handler.cpp:71] Performing capacity heuristic check for a set 
> quota request
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.763783 
> 24902 quota_handler.cpp:87] heuristic: total quota 'disk(*):143489'
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.763870 
> 24902 quota_handler.cpp:111] heuristic: nonStaticAgentResources = 
> 'ports(*):[1025-2180, 2182-3887, 3889-5049, 5052-8079, 8082-8180, 
> 8182-32000]; disk(*)[MOUNT:/dcos/volume0]:7373; 
> disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; 
> disk(*):28698; cpus(*):4; mem(*):15023'
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.763923 
> 24902 quota_handler.cpp:113] heuristic: nonStaticClusterResources = 
> 'ports(*):[1025-2180, 2182-3887, 3889-5049, 5052-8079, 8082-8180, 
> 8182-32000]; disk(*)[MOUNT:/dcos/volume0]:7373; 
> disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; 
> disk(*):28698; cpus(*):4; mem(*):15023'
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.763989 
> 24902 quota_handler.cpp:111] heuristic: nonStaticAgentResources = 
> 'ports(*):[1025-2180, 2182-3887, 3889-5049, 5052-8079, 8082-8180, 
> 8182-32000]; disk(*)[MOUNT:/dcos/volume0]:7373; 
> disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; 
> disk(*):28698; cpus(*):4; mem(*):15023'
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.764022 
> 24902 quota_handler.cpp:113] heuristic: nonStaticClusterResources = 
> 'ports(*):[1025-2180, 2182-3887, 3889-5049, 5052-8079, 8082-8180, 
> 8182-32000]; disk(*)[MOUNT:/dcos/volume0]:7373; 
> disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; 
> disk(*):57396; cpus(*):8; mem(*):30046; disk(*)[MOUNT:/dcos/volume0]:7373; 
> disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373'
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.764077 
> 24902 quota_handler.cpp:111] heuristic: nonStaticAgentResources = 
> 'ports(*):[1025-2180, 2182-3887, 3889-5049, 5052-8079, 8082-8180, 
> 8182-32000]; disk(*)[MOUNT:/dcos/volume0]:7373; 
> disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; 
> disk(*):28695; cpus(*):4; mem(*):15023'
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.764119 
> 24902 quota_handler.cpp:113] heuristic: nonStaticClusterResources = 
> 'ports(*):[1025-2180, 2182-3887, 3889-5049, 5052-8079, 8082-8180, 
> 8182-32000]; disk(*)[MOUNT:/dcos/volume0]:7373; 
> disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; 
> disk(*):86091; cpus(*):12; mem(*):45069; disk(*)[MOUNT:/dcos/volume0]:7373; 
> disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; 
> disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; 
> disk(*)[MOUNT:/dcos/volume2]:7373'
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.764225 
> 24902 quota_handler.cpp:111] heuristic: nonStaticAgentResources = 
> 'ports(*):[1025-2180, 2182-3887, 3889-5049, 5052-8079, 8082-8180, 
> 8182-32000]; disk(*)[MOUNT:/dcos/volume0]:7373; 
> disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; 
> disk(*):28700; cpus(*):4; mem(*):15023'
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.764307 
> 24902 quota_handler.cpp:113] heuristic: nonStaticClusterResources = 
> 'ports(*):[1025-2180, 2182-3887, 3889-5049, 5052-8079, 8082-8180, 
> 8182-32000]; disk(*)[MOUNT:/dcos/volume0]:7373; 
> disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; 
> disk(*):114791; cpus(*):16; mem(*):60092; disk(*)[MOUNT:/dcos/volume0]:7373; 
> disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; 
> disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; 
> disk(*)[MOUNT:/dcos/volume2]:7373; disk(*)[MOUNT:/dcos/volume0]:7373; 
> disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373'
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.764434 
> 24902 quota_handler.cpp:111] heuristic: nonStaticAgentResources = 
> 'ports(*):[1025-2180, 2182-3887, 3889-5049, 5052-8079, 8082-8180, 
> 8182-32000]; disk(*)[MOUNT:/dcos/volume0]:7373; 
> disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; 
> disk(*):28698; cpus(*):4; mem(*):15023'
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.764492 
> 24902 quota_handler.cpp:113] heuristic: nonStaticClusterResources = 
> 'ports(*):[1025-2180, 2182-3887, 3889-5049, 5052-8079, 8082-8180, 
> 8182-32000]; disk(*)[MOUNT:/dcos/volume0]:7373; 
> disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; 
> disk(*):143489; cpus(*):20; mem(*):75115; disk(*)[MOUNT:/dcos/volume0]:7373; 
> disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; 
> disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; 
> disk(*)[MOUNT:/dcos/volume2]:7373; disk(*)[MOUNT:/dcos/volume0]:7373; 
> disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; 
> disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; 
> disk(*)[MOUNT:/dcos/volume2]:7373'
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.764562 
> 24902 quota_handler.cpp:118] heuristic: 
> nonStaticClusterResources.contains(totalQuota)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to