See <https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Disable-Java-Disable-Python-Disable-Webui/292/changes>
Changes: [bmahler] Fixed MonitorTest.WatchUnwatch flakiness by adding missing Clock::settle calls. Review: https://reviews.apache.org/r/10123 [bmahler] Fixed Statistics.archive test flakiness by adding a call to Clock::settle. Review: https://reviews.apache.org/r/10120 ------------------------------------------ [...truncated 7952 lines...] I0329 03:27:44.533043 8333 exec.cpp:258] Executor asked to run task '1' I0329 03:27:44.533108 8333 exec.cpp:382] Executor sending status update for task 1 in state TASK_RUNNING I0329 03:27:44.534065 8333 slave.cpp:1232] Handling status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.534119 8333 slave.cpp:1280] Forwarding status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 to the status update manager I0329 03:27:44.534252 8333 status_update_manager.cpp:254] Received status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.534291 8333 status_update_manager.cpp:403] Creating StatusUpdate stream for task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.534339 8333 status_update_manager.hpp:314] Handling UPDATE for status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.534384 8333 status_update_manager.cpp:289] Forwarding status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 to the master at [email protected]:42543 I0329 03:27:44.534482 8333 slave.cpp:1342] Sending ACK for status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 to executor executor(15)@67.195.138.9:42543 I0329 03:27:44.534490 8326 master.cpp:1086] Status update from (189)@67.195.138.9:42543: task 1 of framework 201303290327-160088899-42543-8291-0000 is now in state TASK_RUNNING I0329 03:27:44.534638 8331 exec.cpp:289] Executor received ACK for status update of task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.534869 8332 sched.cpp:182] New master at [email protected]:42543 I0329 03:27:44.534977 8332 master.cpp:681] Re-registering framework 201303290327-160088899-42543-8291-0000 at scheduler(44)@67.195.138.9:42543 I0329 03:27:44.535033 8332 master.cpp:700] Framework 201303290327-160088899-42543-8291-0000 failed over I0329 03:27:44.535174 8328 sched.cpp:413] Got error 'Framework failed over' I0329 03:27:44.535178 8330 slave.cpp:956] Updating framework 201303290327-160088899-42543-8291-0000 pid to scheduler(44)@67.195.138.9:42543 I0329 03:27:44.535265 8328 sched.cpp:446] Aborting framework '201303290327-160088899-42543-8291-0000' I0329 03:27:44.535182 8326 sched.cpp:217] Framework registered with 201303290327-160088899-42543-8291-0000 W0329 03:27:44.535352 8328 master.cpp:798] scheduler(43)@67.195.138.9:42543 tried to deactivate framework; expecting scheduler(44)@67.195.138.9:42543 I0329 03:27:44.535578 8332 hierarchical_allocator_process.hpp:668] No resources available to allocate! W0329 03:27:44.535616 8329 master.cpp:81] No whitelist given. Advertising offers for all slaves I0329 03:27:44.535853 8329 monitor.cpp:206] Publishing resource usage for executor 'default' of framework '201303290327-160088899-42543-8291-0000' I0329 03:27:44.535929 8327 status_update_manager.cpp:379] Checking for unacknowledged status updates W0329 03:27:44.535961 8327 status_update_manager.cpp:388] Resending status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.536015 8327 status_update_manager.cpp:289] Forwarding status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 to the master at [email protected]:42543 I0329 03:27:44.535651 8332 hierarchical_allocator_process.hpp:599] Performed allocation for 1 slaves in 80.48us I0329 03:27:44.536118 8333 master.cpp:1086] Status update from (189)@67.195.138.9:42543: task 1 of framework 201303290327-160088899-42543-8291-0000 is now in state TASK_RUNNING I0329 03:27:44.536249 8333 sched.cpp:327] Received status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 from slave(56)@67.195.138.9:42543 I0329 03:27:44.536321 8333 sched.cpp:360] Sending ACK for status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 to slave(56)@67.195.138.9:42543 I0329 03:27:44.536389 8333 slave.cpp:979] Got acknowledgement of status update for task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.536430 8326 sched.cpp:422] Stopping framework '201303290327-160088899-42543-8291-0000' I0329 03:27:44.536408 8327 sched.cpp:422] Stopping framework '201303290327-160088899-42543-8291-0000' I0329 03:27:44.536545 8328 master.cpp:774] Asked to unregister framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.536525 8333 status_update_manager.cpp:314] Received status update acknowledgement for task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.536638 8326 hierarchical_allocator_process.hpp:359] Deactivated framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.536659 8333 status_update_manager.hpp:314] Handling ACK for status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.536663 8330 slave.cpp:889] Asked to shut down framework 201303290327-160088899-42543-8291-0000 by [email protected]:42543 I0329 03:27:44.536658 8328 master.hpp:300] Removing task with resources cpus=2; mem=1024; ports=[31000-32000]; disk=1024 on slave 201303290327-160088899-42543-8291-0 I0329 03:27:44.536792 8330 slave.cpp:894] Shutting down framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.536880 8330 slave.cpp:1681] Shutting down executor 'default' of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.536928 8328 master.cpp:774] Asked to unregister framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.536939 8330 slave.cpp:1016] Status update manager successfully handled status update acknowledgement for task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.537022 8330 exec.cpp:321] Executor asked to shutdown I0329 03:27:44.537045 8328 hierarchical_allocator_process.hpp:544] Recovered cpus=2; mem=1024; ports=[31000-32000]; disk=1024 (total allocatable: cpus=2; mem=1024; ports=[31000-32000]; disk=1024) on slave 201303290327-160088899-42543-8291-0 from framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.537070 8330 status_update_manager.cpp:233] Closing status update streams for framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.537145 8330 status_update_manager.cpp:434] Cleaning up status update stream for task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.537190 8291 slave.cpp:386] Slave terminating I0329 03:27:44.537227 8291 slave.cpp:889] Asked to shut down framework 201303290327-160088899-42543-8291-0000 by @0.0.0.0:0 I0329 03:27:44.537256 8291 slave.cpp:894] Shutting down framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.537279 8291 slave.cpp:1681] Shutting down executor 'default' of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.537338 8291 master.cpp:477] Master terminating I0329 03:27:44.537348 8330 status_update_manager.cpp:233] Closing status update streams for framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.537621 8291 master.cpp:283] Shutting down master I0329 03:27:44.537147 8328 hierarchical_allocator_process.hpp:312] Removed framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.537832 8328 hierarchical_allocator_process.hpp:423] Removed slave 201303290327-160088899-42543-8291-0 [ OK ] FaultToleranceTest.SchedulerFailoverStatusUpdate (11 ms) [ RUN ] FaultToleranceTest.ForwardStatusUpdateUnknownExecutor I0329 03:27:44.539711 8333 master.cpp:309] Master started on 67.195.138.9:42543 I0329 03:27:44.539800 8333 master.cpp:324] Master ID: 201303290327-160088899-42543-8291 W0329 03:27:44.539983 8328 master.cpp:81] No whitelist given. Advertising offers for all slaves I0329 03:27:44.540017 8329 hierarchical_allocator_process.hpp:236] Initializing hierarchical allocator process with master : [email protected]:42543 I0329 03:27:44.540151 8328 slave.cpp:203] Slave started on 57)@67.195.138.9:42543 I0329 03:27:44.540197 8328 slave.cpp:204] Slave resources: cpus=2; mem=1024; ports=[31000-32000]; disk=1024 I0329 03:27:44.540210 8333 master.cpp:603] Elected as master! I0329 03:27:44.540350 8329 sched.cpp:182] New master at [email protected]:42543 I0329 03:27:44.540457 8327 master.cpp:646] Registering framework 201303290327-160088899-42543-8291-0000 at scheduler(45)@67.195.138.9:42543 I0329 03:27:44.540526 8327 hierarchical_allocator_process.hpp:268] Added framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.540557 8327 hierarchical_allocator_process.hpp:668] No resources available to allocate! I0329 03:27:44.540557 8332 sched.cpp:217] Framework registered with 201303290327-160088899-42543-8291-0000 I0329 03:27:44.540588 8328 slave.cpp:453] New master detected at [email protected]:42543 I0329 03:27:44.540587 8327 hierarchical_allocator_process.hpp:599] Performed allocation for 0 slaves in 30.55us I0329 03:27:44.540727 8328 slave.cpp:377] Finished recovery I0329 03:27:44.540745 8326 status_update_manager.cpp:132] New master detected at [email protected]:42543 I0329 03:27:44.540966 8327 master.cpp:968] Attempting to register slave on vesta.apache.org at slave(57)@67.195.138.9:42543 I0329 03:27:44.541013 8327 master.cpp:1224] Master now considering a slave at vesta.apache.org:42543 as active I0329 03:27:44.541051 8327 master.cpp:1862] Adding slave 201303290327-160088899-42543-8291-0 at vesta.apache.org with cpus=2; mem=1024; ports=[31000-32000]; disk=1024 I0329 03:27:44.541131 8333 slave.cpp:487] Registered with master; given slave ID 201303290327-160088899-42543-8291-0 I0329 03:27:44.541184 8327 hierarchical_allocator_process.hpp:395] Added slave 201303290327-160088899-42543-8291-0 (vesta.apache.org) with cpus=2; mem=1024; ports=[31000-32000]; disk=1024 (and cpus=2; mem=1024; ports=[31000-32000]; disk=1024 available) I0329 03:27:44.541265 8327 hierarchical_allocator_process.hpp:660] Found available resources: cpus=2; mem=1024; ports=[31000-32000]; disk=1024 on slave 201303290327-160088899-42543-8291-0 I0329 03:27:44.541309 8327 hierarchical_allocator_process.hpp:686] Offering cpus=2; mem=1024; ports=[31000-32000]; disk=1024 on slave 201303290327-160088899-42543-8291-0 to framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.541425 8327 hierarchical_allocator_process.hpp:619] Performed allocation for slave 201303290327-160088899-42543-8291-0 in 167.01us I0329 03:27:44.541468 8333 master.hpp:309] Adding offer with resources cpus=2; mem=1024; ports=[31000-32000]; disk=1024 on slave 201303290327-160088899-42543-8291-0 I0329 03:27:44.541554 8333 master.cpp:1327] Sending 1 offers to framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.541621 8333 sched.cpp:282] Received 1 offers I0329 03:27:44.541853 8327 master.cpp:1534] Processing reply for offer 201303290327-160088899-42543-8291-0 on slave 201303290327-160088899-42543-8291-0 (vesta.apache.org) for framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.541996 8327 master.hpp:289] Adding task with resources cpus=2; mem=1024; ports=[31000-32000]; disk=1024 on slave 201303290327-160088899-42543-8291-0 I0329 03:27:44.542057 8327 master.cpp:1651] Launching task 1 of framework 201303290327-160088899-42543-8291-0000 with resources cpus=2; mem=1024; ports=[31000-32000]; disk=1024 on slave 201303290327-160088899-42543-8291-0 (vesta.apache.org) I0329 03:27:44.542171 8332 slave.cpp:587] Got assigned task 1 for framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.542211 8327 master.hpp:318] Removing offer with resources cpus=2; mem=1024; ports=[31000-32000]; disk=1024 on slave 201303290327-160088899-42543-8291-0 I0329 03:27:44.543267 8332 paths.hpp:302] Created executor directory '/tmp/FaultToleranceTest_ForwardStatusUpdateUnknownExecutor_vbZklZ/slaves/201303290327-160088899-42543-8291-0/frameworks/201303290327-160088899-42543-8291-0000/executors/default/runs/dfe361ce-8eef-4907-aa54-00bf4112d6c5' I0329 03:27:44.543547 8329 slave.cpp:436] Successfully attached file '/tmp/FaultToleranceTest_ForwardStatusUpdateUnknownExecutor_vbZklZ/slaves/201303290327-160088899-42543-8291-0/frameworks/201303290327-160088899-42543-8291-0000/executors/default/runs/dfe361ce-8eef-4907-aa54-00bf4112d6c5' I0329 03:27:44.543616 8332 exec.cpp:170] Executor started at: executor(16)@67.195.138.9:42543 with pid 8291 I0329 03:27:44.543684 8332 slave.cpp:1046] Got registration for executor 'default' of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.543824 8332 slave.cpp:1121] Flushing queued tasks for framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.543850 8329 exec.cpp:194] Executor registered on slave 201303290327-160088899-42543-8291-0 I0329 03:27:44.543946 8329 exec.cpp:258] Executor asked to run task '1' I0329 03:27:44.544000 8329 exec.cpp:382] Executor sending status update for task 1 in state TASK_RUNNING I0329 03:27:44.544870 8331 slave.cpp:1232] Handling status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.544914 8331 slave.cpp:1280] Forwarding status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 to the status update manager I0329 03:27:44.545017 8331 status_update_manager.cpp:254] Received status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.545058 8331 status_update_manager.cpp:403] Creating StatusUpdate stream for task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.545105 8331 status_update_manager.hpp:314] Handling UPDATE for status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.545146 8331 status_update_manager.cpp:289] Forwarding status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 to the master at [email protected]:42543 I0329 03:27:44.545219 8329 master.cpp:1086] Status update from (193)@67.195.138.9:42543: task 1 of framework 201303290327-160088899-42543-8291-0000 is now in state TASK_RUNNING I0329 03:27:44.545223 8331 slave.cpp:1342] Sending ACK for status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 to executor executor(16)@67.195.138.9:42543 I0329 03:27:44.545300 8329 sched.cpp:327] Received status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 from slave(57)@67.195.138.9:42543 I0329 03:27:44.545331 8331 exec.cpp:289] Executor received ACK for status update of task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.545368 8329 sched.cpp:360] Sending ACK for status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 to slave(57)@67.195.138.9:42543 I0329 03:27:44.545436 8329 slave.cpp:979] Got acknowledgement of status update for task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.545497 8329 status_update_manager.cpp:314] Received status update acknowledgement for task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.545536 8329 status_update_manager.hpp:314] Handling ACK for status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.545583 8329 slave.cpp:1016] Status update manager successfully handled status update acknowledgement for task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.546213 8329 slave.cpp:1232] Handling status update TASK_RUNNING from task task2 of framework 201303290327-160088899-42543-8291-0000 W0329 03:27:44.546262 8329 slave.cpp:1256] Could not find executor for task task2 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.546300 8329 slave.cpp:1280] Forwarding status update TASK_RUNNING from task task2 of framework 201303290327-160088899-42543-8291-0000 to the status update manager I0329 03:27:44.546378 8329 status_update_manager.cpp:254] Received status update TASK_RUNNING from task task2 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.546417 8329 status_update_manager.cpp:403] Creating StatusUpdate stream for task task2 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.546447 8329 status_update_manager.hpp:314] Handling UPDATE for status update TASK_RUNNING from task task2 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.546478 8329 status_update_manager.cpp:289] Forwarding status update TASK_RUNNING from task task2 of framework 201303290327-160088899-42543-8291-0000 to the master at [email protected]:42543 I0329 03:27:44.546641 8330 master.cpp:1086] Status update from (193)@67.195.138.9:42543: task task2 of framework 201303290327-160088899-42543-8291-0000 is now in state TASK_RUNNING W0329 03:27:44.546813 8330 master.cpp:1119] Status update from (193)@67.195.138.9:42543 (vesta.apache.org): error, couldn't lookup task task2 I0329 03:27:44.546875 8329 sched.cpp:327] Received status update TASK_RUNNING from task task2 of framework 201303290327-160088899-42543-8291-0000 from slave(57)@67.195.138.9:42543 I0329 03:27:44.546962 8329 sched.cpp:360] Sending ACK for status update TASK_RUNNING from task task2 of framework 201303290327-160088899-42543-8291-0000 to slave(57)@67.195.138.9:42543 I0329 03:27:44.547014 8329 sched.cpp:422] Stopping framework '201303290327-160088899-42543-8291-0000' I0329 03:27:44.547083 8332 master.cpp:774] Asked to unregister framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.547021 8327 slave.cpp:979] Got acknowledgement of status update for task task2 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.547224 8332 master.hpp:300] Removing task with resources cpus=2; mem=1024; ports=[31000-32000]; disk=1024 on slave 201303290327-160088899-42543-8291-0 I0329 03:27:44.547282 8327 slave.cpp:889] Asked to shut down framework 201303290327-160088899-42543-8291-0000 by [email protected]:42543 I0329 03:27:44.547286 8329 status_update_manager.cpp:314] Received status update acknowledgement for task task2 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.547204 8330 hierarchical_allocator_process.hpp:359] Deactivated framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.547373 8327 slave.cpp:894] Shutting down framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.547394 8329 status_update_manager.hpp:314] Handling ACK for status update TASK_RUNNING from task task2 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.547531 8327 slave.cpp:1681] Shutting down executor 'default' of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.547565 8330 hierarchical_allocator_process.hpp:544] Recovered cpus=2; mem=1024; ports=[31000-32000]; disk=1024 (total allocatable: cpus=2; mem=1024; ports=[31000-32000]; disk=1024) on slave 201303290327-160088899-42543-8291-0 from framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.547701 8333 exec.cpp:321] Executor asked to shutdown I0329 03:27:44.547811 8330 hierarchical_allocator_process.hpp:312] Removed framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.547711 8329 status_update_manager.cpp:233] Closing status update streams for framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.547711 8327 slave.cpp:1016] Status update manager successfully handled status update acknowledgement for task task2 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.547967 8329 status_update_manager.cpp:434] Cleaning up status update stream for task task2 of framework 201303290327-160088899-42543-8291-0000 E0329 03:27:44.548018 8327 slave.cpp:1030] Status update acknowledgement for task task2 of unknown executor I0329 03:27:44.548068 8329 status_update_manager.cpp:434] Cleaning up status update stream for task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.548140 8327 slave.cpp:386] Slave terminating I0329 03:27:44.548209 8327 slave.cpp:889] Asked to shut down framework 201303290327-160088899-42543-8291-0000 by @0.0.0.0:0 I0329 03:27:44.548239 8327 slave.cpp:894] Shutting down framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.548265 8327 slave.cpp:1681] Shutting down executor 'default' of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.548327 8326 status_update_manager.cpp:233] Closing status update streams for framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.548357 8332 master.cpp:537] Slave 201303290327-160088899-42543-8291-0(vesta.apache.org) disconnected I0329 03:27:44.548462 8332 master.cpp:542] Removing disconnected slave 201303290327-160088899-42543-8291-0(vesta.apache.org) because it is not checkpointing! I0329 03:27:44.548591 8332 master.cpp:477] Master terminating I0329 03:27:44.548622 8327 hierarchical_allocator_process.hpp:423] Removed slave 201303290327-160088899-42543-8291-0 I0329 03:27:44.548945 8291 master.cpp:283] Shutting down master [ OK ] FaultToleranceTest.ForwardStatusUpdateUnknownExecutor (11 ms) [ RUN ] FaultToleranceTest.SchedulerFailoverFrameworkMessage I0329 03:27:44.551334 8330 master.cpp:309] Master started on 67.195.138.9:42543 I0329 03:27:44.551424 8330 master.cpp:324] Master ID: 201303290327-160088899-42543-8291 W0329 03:27:44.551749 8328 master.cpp:81] No whitelist given. Advertising offers for all slaves I0329 03:27:44.551774 8326 slave.cpp:203] Slave started on 58)@67.195.138.9:42543 I0329 03:27:44.551818 8331 hierarchical_allocator_process.hpp:236] Initializing hierarchical allocator process with master : [email protected]:42543 I0329 03:27:44.551897 8326 slave.cpp:204] Slave resources: cpus=2; mem=1024; ports=[31000-32000]; disk=1024 I0329 03:27:44.551980 8330 master.cpp:603] Elected as master! I0329 03:27:44.552032 8328 sched.cpp:182] New master at [email protected]:42543 I0329 03:27:44.552366 8328 master.cpp:646] Registering framework 201303290327-160088899-42543-8291-0000 at scheduler(46)@67.195.138.9:42543 I0329 03:27:44.552466 8328 hierarchical_allocator_process.hpp:268] Added framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.552467 8329 sched.cpp:217] Framework registered with 201303290327-160088899-42543-8291-0000 I0329 03:27:44.552502 8328 hierarchical_allocator_process.hpp:668] No resources available to allocate! I0329 03:27:44.552522 8326 slave.cpp:453] New master detected at [email protected]:42543 I0329 03:27:44.552644 8328 hierarchical_allocator_process.hpp:599] Performed allocation for 0 slaves in 140.07us I0329 03:27:44.552716 8326 slave.cpp:377] Finished recovery I0329 03:27:44.552729 8327 status_update_manager.cpp:132] New master detected at [email protected]:42543 I0329 03:27:44.553004 8327 master.cpp:968] Attempting to register slave on vesta.apache.org at slave(58)@67.195.138.9:42543 I0329 03:27:44.553036 8327 master.cpp:1224] Master now considering a slave at vesta.apache.org:42543 as active I0329 03:27:44.553067 8327 master.cpp:1862] Adding slave 201303290327-160088899-42543-8291-0 at vesta.apache.org with cpus=2; mem=1024; ports=[31000-32000]; disk=1024 I0329 03:27:44.553143 8329 slave.cpp:487] Registered with master; given slave ID 201303290327-160088899-42543-8291-0 I0329 03:27:44.553220 8326 hierarchical_allocator_process.hpp:395] Added slave 201303290327-160088899-42543-8291-0 (vesta.apache.org) with cpus=2; mem=1024; ports=[31000-32000]; disk=1024 (and cpus=2; mem=1024; ports=[31000-32000]; disk=1024 available) I0329 03:27:44.553356 8326 hierarchical_allocator_process.hpp:660] Found available resources: cpus=2; mem=1024; ports=[31000-32000]; disk=1024 on slave 201303290327-160088899-42543-8291-0 I0329 03:27:44.553413 8326 hierarchical_allocator_process.hpp:686] Offering cpus=2; mem=1024; ports=[31000-32000]; disk=1024 on slave 201303290327-160088899-42543-8291-0 to framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.553607 8326 hierarchical_allocator_process.hpp:619] Performed allocation for slave 201303290327-160088899-42543-8291-0 in 259.96us I0329 03:27:44.553665 8327 master.hpp:309] Adding offer with resources cpus=2; mem=1024; ports=[31000-32000]; disk=1024 on slave 201303290327-160088899-42543-8291-0 I0329 03:27:44.553743 8327 master.cpp:1327] Sending 1 offers to framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.553817 8327 sched.cpp:282] Received 1 offers I0329 03:27:44.554059 8330 master.cpp:1534] Processing reply for offer 201303290327-160088899-42543-8291-0 on slave 201303290327-160088899-42543-8291-0 (vesta.apache.org) for framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.554190 8330 master.hpp:289] Adding task with resources cpus=2; mem=1024; ports=[31000-32000]; disk=1024 on slave 201303290327-160088899-42543-8291-0 I0329 03:27:44.554250 8330 master.cpp:1651] Launching task 1 of framework 201303290327-160088899-42543-8291-0000 with resources cpus=2; mem=1024; ports=[31000-32000]; disk=1024 on slave 201303290327-160088899-42543-8291-0 (vesta.apache.org) I0329 03:27:44.554324 8329 slave.cpp:587] Got assigned task 1 for framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.554368 8330 master.hpp:318] Removing offer with resources cpus=2; mem=1024; ports=[31000-32000]; disk=1024 on slave 201303290327-160088899-42543-8291-0 I0329 03:27:44.555361 8329 paths.hpp:302] Created executor directory '/tmp/FaultToleranceTest_SchedulerFailoverFrameworkMessage_lKnpUW/slaves/201303290327-160088899-42543-8291-0/frameworks/201303290327-160088899-42543-8291-0000/executors/default/runs/b2fdbb22-4d59-4d84-a35e-e2628f18ac52' I0329 03:27:44.555551 8330 slave.cpp:436] Successfully attached file '/tmp/FaultToleranceTest_SchedulerFailoverFrameworkMessage_lKnpUW/slaves/201303290327-160088899-42543-8291-0/frameworks/201303290327-160088899-42543-8291-0000/executors/default/runs/b2fdbb22-4d59-4d84-a35e-e2628f18ac52' I0329 03:27:44.555587 8329 exec.cpp:170] Executor started at: executor(17)@67.195.138.9:42543 with pid 8291 I0329 03:27:44.555631 8329 slave.cpp:1046] Got registration for executor 'default' of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.555738 8329 slave.cpp:1121] Flushing queued tasks for framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.555773 8326 exec.cpp:194] Executor registered on slave 201303290327-160088899-42543-8291-0 I0329 03:27:44.555872 8326 exec.cpp:258] Executor asked to run task '1' I0329 03:27:44.555927 8326 exec.cpp:382] Executor sending status update for task 1 in state TASK_RUNNING I0329 03:27:44.556794 8326 slave.cpp:1232] Handling status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.556833 8326 slave.cpp:1280] Forwarding status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 to the status update manager I0329 03:27:44.556964 8331 status_update_manager.cpp:254] Received status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.557006 8331 status_update_manager.cpp:403] Creating StatusUpdate stream for task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.557046 8331 status_update_manager.hpp:314] Handling UPDATE for status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.557096 8331 status_update_manager.cpp:289] Forwarding status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 to the master at [email protected]:42543 I0329 03:27:44.557191 8326 master.cpp:1086] Status update from (197)@67.195.138.9:42543: task 1 of framework 201303290327-160088899-42543-8291-0000 is now in state TASK_RUNNING I0329 03:27:44.557339 8326 sched.cpp:327] Received status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 from slave(58)@67.195.138.9:42543 I0329 03:27:44.557384 8327 slave.cpp:1342] Sending ACK for status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 to executor executor(17)@67.195.138.9:42543 I0329 03:27:44.557414 8326 sched.cpp:360] Sending ACK for status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 to slave(58)@67.195.138.9:42543 I0329 03:27:44.557456 8327 exec.cpp:289] Executor received ACK for status update of task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.557528 8328 slave.cpp:979] Got acknowledgement of status update for task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.557646 8333 sched.cpp:182] New master at [email protected]:42543 I0329 03:27:44.557677 8328 status_update_manager.cpp:314] Received status update acknowledgement for task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.557734 8333 master.cpp:681] Re-registering framework 201303290327-160088899-42543-8291-0000 at scheduler(47)@67.195.138.9:42543 I0329 03:27:44.557888 8333 master.cpp:700] Framework 201303290327-160088899-42543-8291-0000 failed over I0329 03:27:44.557816 8328 status_update_manager.hpp:314] Handling ACK for status update TASK_RUNNING from task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.557991 8330 sched.cpp:217] Framework registered with 201303290327-160088899-42543-8291-0000 I0329 03:27:44.558001 8331 slave.cpp:956] Updating framework 201303290327-160088899-42543-8291-0000 pid to scheduler(47)@67.195.138.9:42543 I0329 03:27:44.557963 8333 sched.cpp:413] Got error 'Framework failed over' I0329 03:27:44.558107 8331 slave.cpp:1016] Status update manager successfully handled status update acknowledgement for task 1 of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.558161 8333 sched.cpp:446] Aborting framework '201303290327-160088899-42543-8291-0000' I0329 03:27:44.558233 8331 slave.cpp:1370] Sending message for framework 201303290327-160088899-42543-8291-0000 to scheduler(47)@67.195.138.9:42543 W0329 03:27:44.558285 8333 master.cpp:798] scheduler(46)@67.195.138.9:42543 tried to deactivate framework; expecting scheduler(47)@67.195.138.9:42543 I0329 03:27:44.558359 8331 sched.cpp:401] Received framework message I0329 03:27:44.558532 8330 sched.cpp:422] Stopping framework '201303290327-160088899-42543-8291-0000' I0329 03:27:44.558537 8291 slave.cpp:386] Slave terminating I0329 03:27:44.558574 8332 sched.cpp:422] Stopping framework '201303290327-160088899-42543-8291-0000' I0329 03:27:44.558619 8330 master.cpp:774] Asked to unregister framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.558641 8291 slave.cpp:889] Asked to shut down framework 201303290327-160088899-42543-8291-0000 by @0.0.0.0:0 W0329 03:27:44.558823 8330 master.cpp:781] scheduler(46)@67.195.138.9:42543 tried to unregister framework; expecting scheduler(47)@67.195.138.9:42543 I0329 03:27:44.558861 8291 slave.cpp:894] Shutting down framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.558897 8330 master.cpp:774] Asked to unregister framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.558917 8291 slave.cpp:1681] Shutting down executor 'default' of framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.558995 8330 master.hpp:300] Removing task with resources cpus=2; mem=1024; ports=[31000-32000]; disk=1024 on slave 201303290327-160088899-42543-8291-0 I0329 03:27:44.559100 8330 master.cpp:477] Master terminating I0329 03:27:44.559142 8330 hierarchical_allocator_process.hpp:359] Deactivated framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.559245 8330 hierarchical_allocator_process.hpp:544] Recovered cpus=2; mem=1024; ports=[31000-32000]; disk=1024 (total allocatable: cpus=2; mem=1024; ports=[31000-32000]; disk=1024) on slave 201303290327-160088899-42543-8291-0 from framework 201303290327-160088899-42543-8291-0000 I0329 03:27:44.559309 8291 master.cpp:283] Shutting down master I0329 03:27:44.559357 8329 exec.cpp:321] Executor asked to shutdown pure virtual method called terminate called without an active exception /bin/bash: line 5: 8291 Aborted ${dir}$tst FAIL: mesos-tests ================== 1 of 1 test failed ================== make[3]: *** [check-TESTS] Error 1 make[3]: Leaving directory `<https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Disable-Java-Disable-Python-Disable-Webui/ws/build/src'> make[2]: *** [check-am] Error 2 make[2]: Leaving directory `<https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Disable-Java-Disable-Python-Disable-Webui/ws/build/src'> make[1]: *** [check] Error 2 make[1]: Leaving directory `<https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Disable-Java-Disable-Python-Disable-Webui/ws/build/src'> make: *** [check-recursive] Error 1 Process leaked file descriptors. See http://wiki.jenkins-ci.org/display/JENKINS/Spawning+processes+from+build for more information Build step 'Execute shell' marked build as failure
