Benno Evers created MESOS-9644:
----------------------------------
Summary: Marking an Agent as Gone Breaks Metrics Process in Unit
Tests
Key: MESOS-9644
URL: https://issues.apache.org/jira/browse/MESOS-9644
Project: Mesos
Issue Type: Bug
Reporter: Benno Evers
When an agent is marked as gone, the master will tell that agent to shut down
which it tries via
{noformat}
// slave.cpp:974
terminate(self());
{noformat}
However, terminating a process will only call `Slave::finalize()`, but *not*
the destructor `Slave::~Slave()`.
In a standalone slave, this doesn't matter since terminating the slave process
will cause the OS process to immediately exit as well.
However, in unit tests that is not the case, and since the slave was never
properly destructed its metrics keys are still contained in the global metrics
object. The pull gauges will then cause a deadlock the next time a metrics
snapshot is requested, since their dispatches will be silently (for VLOG < 2)
dropped:
{noformat}
I0311 11:08:53.329043 34499 authorization.cpp:135] Authorizing principal 'ANY'
to GET the endpoint '/metrics/snapshot'
I0311 11:08:53.329067 34499 clock.cpp:435] Clock of
local-authorizer(2)@66.70.182.167:35815 updated to 2019-03-11
15:08:53.273557888+00:00
I0311 11:08:53.329121 34499 process.cpp:2880] Resuming
local-authorizer(2)@66.70.182.167:35815 at 2019-03-11 15:08:53.273557888+00:00
I0311 11:08:53.329160 34496 process.cpp:2880] Resuming
__auth_handlers__(2)@66.70.182.167:35815 at 2019-03-11 15:08:53.273557888+00:00
I0311 11:08:53.329260 34496 process.cpp:2880] Resuming
[email protected]:35815 at 2019-03-11 15:08:53.273557888+00:00
I0311 11:08:53.353018 34486 process.cpp:2803] Dropping event for process
slave(1)@66.70.182.167:35815
I0311 11:08:53.353040 34486 process.cpp:2803] Dropping event for process
slave(1)@66.70.182.167:35815
I0311 11:08:53.353063 34486 process.cpp:2803] Dropping event for process
slave(1)@66.70.182.167:35815
I0311 11:08:53.353080 34486 process.cpp:2803] Dropping event for process
slave(1)@66.70.182.167:35815
I0311 11:08:53.353097 34486 process.cpp:2803] Dropping event for process
slave(1)@66.70.182.167:35815
[...]
{noformat}
It's not immediately clear to me what the correct fix for this would be.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
