Repository: mesos Updated Branches: refs/heads/1.4.x e1654ac48 -> 6144242b8
Fixed a crash when metrics race with quota removal. This patch addresses a race condition in which the removal of a role from the allocator's quota sorter races with execution of a callback tied to a `PullGauge`. The gauge's callback assumed that the role would be present in the sorter, but it's possible for the role to be removed before the callback is executed. Review: https://reviews.apache.org/r/67104/ Project: http://git-wip-us.apache.org/repos/asf/mesos/repo Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/6254907c Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/6254907c Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/6254907c Branch: refs/heads/1.4.x Commit: 6254907c15cd0f3a2d0ccd8998377a9fdd81ef54 Parents: e1654ac Author: Greg Mann <[email protected]> Authored: Mon May 14 09:21:26 2018 -0700 Committer: Greg Mann <[email protected]> Committed: Tue May 15 12:18:58 2018 -0700 ---------------------------------------------------------------------- src/master/allocator/mesos/hierarchical.cpp | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/mesos/blob/6254907c/src/master/allocator/mesos/hierarchical.cpp ---------------------------------------------------------------------- diff --git a/src/master/allocator/mesos/hierarchical.cpp b/src/master/allocator/mesos/hierarchical.cpp index e871999..ee7d758 100644 --- a/src/master/allocator/mesos/hierarchical.cpp +++ b/src/master/allocator/mesos/hierarchical.cpp @@ -2427,8 +2427,14 @@ double HierarchicalAllocatorProcess::_quota_allocated( const string& role, const string& resource) { + if (!roleSorter->contains(role)) { + // This can occur when execution of this callback races with removal of the + // metric for a role which does not have any associated frameworks. + return 0.; + } + Option<Value::Scalar> used = - quotaRoleSorter->allocationScalarQuantities(role) + roleSorter->allocationScalarQuantities(role) .get<Value::Scalar>(resource); return used.isSome() ? used->value() : 0;
