-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72509/
-----------------------------------------------------------
(Updated May 14, 2020, 2:42 p.m.)
Review request for mesos and Benjamin Mahler.
Bugs: MESOS-10128
https://issues.apache.org/jira/browse/MESOS-10128
Repository: mesos
Description
-------
Got rid of tracking reserved offered/allocated in roles tree.
Diffs
-----
src/master/allocator/mesos/hierarchical.hpp
6454cdaa19f776365df34ecf83114f0d6fa20f27
src/master/allocator/mesos/hierarchical.cpp
5fe9ffcb518b8427d663ddae43e550795d290e3c
Diff: https://reviews.apache.org/r/72509/diff/1/
Testing (updated)
-------
`make check`
Ran hierarchical allocator benchmarks, compared with current results on master
and 1.9.0 release.
Below are results that improved significantly (compared to 1.10.x branch root)
after this fix.
After fix:
```
[ RUN ]
NonQuotaVsQuotaParam/BENCHMARK_HierarchicalAllocator_WithNonQuotaVsQuotaParam.NonQuotaVsQuota/5
Added 2000 agents in 28.968777ms
Added 2000 frameworks in 3.254205437secs
Quota run setup: 2000 agents, 1000 roles, 2000 frameworks, with drf sorter
Made 2000 allocations in 190.023349ms
Made 0 allocation in 81.404659ms
[ OK ]
NonQuotaVsQuotaParam/BENCHMARK_HierarchicalAllocator_WithNonQuotaVsQuotaParam.NonQuotaVsQuota/5
(3972 ms)
[ RUN ]
AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/48
Using 10000 agents and 1 frameworks with resource sharedness Regular
Added 1 frameworks in 265991ns
Added 10000 agents in 1.088327374secs
round 0 allocate() took 398.34303ms to make 10000 offers
round 1 allocate() took 436.728365ms to make 10000 offers
round 2 allocate() took 431.187768ms to make 10000 offers
round 3 allocate() took 461.27924ms to make 10000 offers
round 4 allocate() took 426.078997ms to make 10000 offers
round 5 allocate() took 478.756617ms to make 10000 offers
[ OK ]
AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/48 (10441
ms)
[ RUN ]
AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/49
Using 10000 agents and 1 frameworks with resource sharedness Shared
Added 1 frameworks in 243347ns
Added 10000 agents in 1.099817273secs
round 0 allocate() took 466.764704ms to make 10000 offers
round 1 allocate() took 466.218437ms to make 10000 offers
round 2 allocate() took 489.643342ms to make 10000 offers
round 3 allocate() took 470.636733ms to make 10000 offers
round 4 allocate() took 441.135047ms to make 10000 offers
round 5 allocate() took 439.438262ms to make 10000 offers
[ OK ]
AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/49 (10693
ms)
...
```
1.10.x/master, before the preceding patch (r72508):
```
[ RUN ]
NonQuotaVsQuotaParam/BENCHMARK_HierarchicalAllocator_WithNonQuotaVsQuotaParam.NonQuotaVsQuota/5
Added 2000 agents in 26.693557ms
Added 2000 frameworks in 3.409660479secs
Quota run setup: 2000 agents, 1000 roles, 2000 frameworks, with drf sorter
Made 2000 allocations in 404.266006ms
Made 0 allocation in 102.640852ms
[ OK ]
NonQuotaVsQuotaParam/BENCHMARK_HierarchicalAllocator_WithNonQuotaVsQuotaParam.NonQuotaVsQuota/5
(4373 ms)
[ RUN ]
AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/48
Using 10000 agents and 1 frameworks with resource sharedness Regular
Added 1 frameworks in 13.767317ms
Added 10000 agents in 18.479510808secs
round 0 allocate() took 15.887923568secs to make 10000 offers
round 1 allocate() took 15.391255543secs to make 10000 offers
round 2 allocate() took 17.560065095secs to make 10000 offers
round 3 allocate() took 15.38378105secs to make 10000 offers
round 4 allocate() took 14.978897854secs to make 10000 offers
```
1.9.0:
```
[ RUN ]
NonQuotaVsQuotaParam/BENCHMARK_HierarchicalAllocator_WithNonQuotaVsQuotaParam.NonQuotaVsQuota/5
Added 2000 agents in 37.441968ms
Added 2000 frameworks in 4.032939233secs
Quota run setup: 2000 agents, 1000 roles, 2000 frameworks, with drf sorter
Made 2000 allocations in 330.437505ms
Made 0 allocation in 232.482817ms
[ OK ]
NonQuotaVsQuotaParam/BENCHMARK_HierarchicalAllocator_WithNonQuotaVsQuotaParam.NonQuotaVsQuota/5
...
[ RUN ]
AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/48
Using 10000 agents and 1 frameworks with resource sharedness Regular
Added 1 frameworks in 21.027254ms
Added 10000 agents in 806.232916ms
round 0 allocate() took 392.39282ms to make 10000 offers
round 1 allocate() took 366.960323ms to make 10000 offers
round 2 allocate() took 369.804604ms to make 10000 offers
round 3 allocate() took 361.787792ms to make 10000 offers
round 4 allocate() took 365.395693ms to make 10000 offers
round 5 allocate() took 394.293667ms to make 10000 offers
[ OK ]
AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/48 (7939
ms)
[ RUN ]
AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/49
Using 10000 agents and 1 frameworks with resource sharedness Shared
Added 1 frameworks in 232664ns
Added 10000 agents in 860.645269ms
round 0 allocate() took 394.275517ms to make 10000 offers
round 1 allocate() took 390.205739ms to make 10000 offers
round 2 allocate() took 391.286642ms to make 10000 offers
round 3 allocate() took 438.51943ms to make 10000 offers
round 4 allocate() took 399.460287ms to make 10000 offers
round 5 allocate() took 439.818163ms to make 10000 offers
[ OK ]
AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/49 (8517
ms)
...
```
Thanks,
Andrei Sekretenko
