[
https://issues.apache.org/jira/browse/MESOS-621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928909#comment-16928909
]
Meng Zhu commented on MESOS-621:
--------------------------------
Added tracking of allocated or offered resources in the allocator:
{noformat}
commit 783fd45c548fdff0c5c4812bc8e92c3aed202e06
Author: Meng Zhu [email protected]
Date: Sat Sep 7 16:01:51 2019 -0700
Tracked offered and allocated resources in the role tree.
This helpers simplify the quota tracking logic and also paves
the way to reduce duplicated states in the sorter.
Also documented that shared resources must be uniquely
identifiable.
Small performance degradation when making allocations due to
duplicated map construction in `(un)trackAllocatedResources`.
This will be removed once embeded the sorter in the role tree.
Benchmark `LargeAndSmallQuota/2`:
Master:
Added 3000 agents in 80.648188ms
Added 3000 frameworks in 19.7006984secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks,
with drf sorter
Made 3500 allocations in 16.044274434secs
Made 0 allocation in 14.476429451secs
Master + this patch:
Added 3000 agents in 80.110817ms
Added 3000 frameworks in 17.25974094secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks,
with drf sorter
Made 3500 allocations in 16.91971379secs
Made 0 allocation in 13.784476154secs
Review: https://reviews.apache.org/r/71460
commit 2ec34ca5951a5a8da3d1ab93839cce68e815c1d5
Author: Meng Zhu <[email protected]>
Date: Tue Sep 3 13:31:36 2019 -0700
Added tracking of framework allocations in the allocator Slave class.
This would simplify the tracking logic regarding
resource allocations in the allocator. See MESOS-9182.
Negligible performance impact:
Master:
BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/2
Added 3000 agents in 77.999483ms
Added 3000 frameworks in 16.736076171secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks,
with drf sorter
Made 3500 allocations in 15.342376944secs
Made 0 allocation in 13.96720191secs
Master + this patch:
BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/2
Added 3000 agents in 83.597048ms
Added 3000 frameworks in 16.757011745secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks,
with drf sorter
Made 3500 allocations in 15.566366241secs
Made 0 allocation in 14.022591871secs
Review: https://reviews.apache.org/r/68508
{noformat}
> `HierarchicalAllocatorProcess::removeSlave` doesn't properly handle framework
> allocations/resources
> ---------------------------------------------------------------------------------------------------
>
> Key: MESOS-621
> URL: https://issues.apache.org/jira/browse/MESOS-621
> Project: Mesos
> Issue Type: Bug
> Components: allocation
> Reporter: Vinod Kone
> Assignee: Meng Zhu
> Priority: Major
> Labels: mesosphere, resource-management, tech-debt
>
> Currently a slaveRemoved() simply removes the slave from 'slaves' map and
> slave's resources from 'roleSorter'. Looking at resourcesRecovered(), more
> things need to be done when a slave is removed (e.g., framework
> unallocations).
> It would be nice to fix this and have a test for this.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
