Benjamin Mahler created MESOS-547:
-------------------------------------
Summary: MasterTest.MasterLost is flaky on Jenkins
Key: MESOS-547
URL: https://issues.apache.org/jira/browse/MESOS-547
Project: Mesos
Issue Type: Bug
Reporter: Benjamin Mahler
Assignee: Benjamin Mahler
This is because the MasterLost test expects the Scheduler.registered call to
happen a single time. However, on the Apache Jenkins machine, it runs slow
enough for the reliable registration in the scheduler driver to retry
registration (1 second timeout).
The fix here will affect a lot of tests, so I'd like to do the right thing,
possible fixes:
EXPECT_CALL(sched, registered(&driver, _, _))
.Times(1); // Change this to .WillRepeatedly(Return())
Also, pausing the clock prior to starting the driver will work. But may have
further impact on the tests.
[ RUN ] MasterTest.MasterLost
I0712 03:48:50.415976 4397 master.cpp:230] Master started on 67.195.138.9:47248
I0712 03:48:50.416594 4399 slave.cpp:113] Slave started on
11)@67.195.138.9:47248
I0712 03:48:50.417034 4395 sched.cpp:177] New master at
[email protected]:47248
I0712 03:48:50.423070 4397 master.cpp:245] Master ID:
201307120348-160088899-47248-4046
I0712 03:48:50.431115 4399 slave.cpp:205] Slave resources: cpus=2; mem=1024;
ports=[31000-32000]; disk=1024
I0712 03:48:50.508512 4398 hierarchical_allocator_process.hpp:305]
Initializing hierarchical allocator process with master :
[email protected]:47248
W0712 03:48:50.508554 4394 master.cpp:83] No whitelist given. Advertising
offers for all slaves
I0712 03:48:50.508767 4397 master.cpp:583] Elected as master!
I0712 03:48:50.791844 4399 slave.cpp:529] New master detected at
[email protected]:47248
I0712 03:48:51.564163 4399 slave.cpp:544] Postponing registration until
recovery is complete
I0712 03:48:51.564257 4396 status_update_manager.cpp:155] New master detected
at [email protected]:47248
I0712 03:48:51.564288 4397 master.cpp:633] Registering framework
201307120348-160088899-47248-4046-0000 at scheduler(11)@67.195.138.9:47248
I0712 03:48:51.751411 4399 slave.cpp:390] Finished recovery
I0712 03:48:51.823364 4392 sched.cpp:222] Framework registered with
201307120348-160088899-47248-4046-0000
I0712 03:48:51.823395 4397 master.cpp:620] Framework
201307120348-160088899-47248-4046-0000 (scheduler(11)@67.195.138.9:47248)
already registered, resending acknowledgement
I0712 03:48:51.823452 4395 hierarchical_allocator_process.hpp:335] Added
framework 201307120348-160088899-47248-4046-0000
I0712 03:48:52.110093 4395 hierarchical_allocator_process.hpp:704] No
resources available to allocate!
I0712 03:48:51.946835 4392 sched.cpp:194] No master detected, waiting for
another master
I0712 03:48:52.110155 4397 master.cpp:962] Attempting to register slave on
vesta.apache.org at slave(11)@67.195.138.9:47248
I0712 03:48:52.440776 4397 master.cpp:1929] Adding slave
201307120348-160088899-47248-4046-0 at vesta.apache.org with cpus=2; mem=1024;
ports=[31000-32000]; disk=1024
I0712 03:48:52.249390 4392 sched.cpp:222] Framework registered with
201307120348-160088899-47248-4046-0000
../../src/tests/master_tests.cpp:906: Failure
Mock function called more times than expected - returning directly.
Function call: registered(0x7fffaf90ca00, @0x2b7bfc004140
201307120348-160088899-47248-4046-0000, @0x2b7bfc003170 40-byte object <70-8B
2C-F5 7B-2B 00-00 00-00 00-00 00-00 00-00 F0-54 00-FC 7B-2B 00-00 43-C3 8A-09
90-B8 00-00 00-00 00-00 07-00 00-00>)
Expected: to be called once
Actual: called twice - over-saturated and active
I0712 03:48:52.241426 4395 hierarchical_allocator_process.hpp:666] Performed
allocation for 0 slaves in 131.314878ms
I0712 03:48:52.448761 4399 slave.cpp:589] Registered with master
[email protected]:47248; given slave ID 201307120348-160088899-47248-4046-0
I0712 03:48:52.448806 4397 master.cpp:436] Master terminating
I0712 03:48:52.481109 4397 slave.cpp:485] Slave asked to shut down by
[email protected]:47248
I0712 03:48:52.489028 4397 slave.cpp:440] Slave terminating
I0712 03:48:52.456776 4392 sched.cpp:427] Stopping framework
'201307120348-160088899-47248-4046-0000'
I0712 03:48:52.481195 4046 master.cpp:209] Shutting down master
I0712 03:48:52.464653 4395 hierarchical_allocator_process.hpp:448] Added slave
201307120348-160088899-47248-4046-0 (vesta.apache.org) with cpus=2; mem=1024;
ports=[31000-32000]; disk=1024 (and cpus=2; mem=1024; ports=[31000-32000];
disk=1024 available)
I0712 03:48:52.560148 4395 hierarchical_allocator_process.hpp:726] Offering
cpus=2; mem=1024; ports=[31000-32000]; disk=1024 on slave
201307120348-160088899-47248-4046-0 to framework
201307120348-160088899-47248-4046-0000
I0712 03:48:52.568372 4395 hierarchical_allocator_process.hpp:686] Performed
allocation for slave 201307120348-160088899-47248-4046-0 in 8.255916ms
[ FAILED ] MasterTest.MasterLost (2161 ms)
[----------] 11 tests from MasterTest (15502 ms total)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
