Preparing for a fix: https://issues.apache.org/jira/browse/MESOS-668
-- Jiang Yan Xu <[email protected]> @xujyan <http://twitter.com/xujyan> On Mon, Aug 26, 2013 at 4:24 PM, Apache Jenkins Server < [email protected]> wrote: > See < > https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-In-Src-Set-JAVA_HOME/1344/changes > > > > Changes: > > [vinod] Added a slave recovery test for the case of multiple slaves. > > [bmahler] Added a recovery timeout for executor driver self-termination. > > ------------------------------------------ > [...truncated 16757 lines...] > I0826 23:24:04.461639 31446 exec.cpp:210] Executor::registered took > 14.142us > I0826 23:24:04.461752 31446 exec.cpp:285] Executor asked to run task '0' > I0826 23:24:04.462594 31446 exec.cpp:294] Executor::launchTask took > 23.971us > I0826 23:24:04.464470 31446 exec.cpp:501] Executor sending status update > TASK_RUNNING (UUID: 4e914b5c-8a53-4ea0-acea-eea9f451b378) for task 0 of > framework 201308262324-1015726915-53338-31395-0000 > I0826 23:24:04.464617 31446 slave.cpp:1772] Handling status update > TASK_RUNNING (UUID: 4e914b5c-8a53-4ea0-acea-eea9f451b378) for task 0 of > framework 201308262324-1015726915-53338-31395-0000 from executor(40)@ > 67.195.138.60:53338 > I0826 23:24:04.470577 31446 status_update_manager.cpp:300] Received status > update TASK_RUNNING (UUID: 4e914b5c-8a53-4ea0-acea-eea9f451b378) for task 0 > of framework 201308262324-1015726915-53338-31395-0000 > I0826 23:24:04.470808 31446 status_update_manager.cpp:471] Creating > StatusUpdate stream for task 0 of framework > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:04.471273 31446 status_update_manager.cpp:351] Forwarding > status update TASK_RUNNING (UUID: 4e914b5c-8a53-4ea0-acea-eea9f451b378) for > task 0 of framework 201308262324-1015726915-53338-31395-0000 to > [email protected]:53338 > I0826 23:24:04.471812 31444 master.cpp:1208] Status update TASK_RUNNING > (UUID: 4e914b5c-8a53-4ea0-acea-eea9f451b378) for task 0 of framework > 201308262324-1015726915-53338-31395-0000 from slave(131)@ > 67.195.138.60:53338 > I0826 23:24:04.471920 31446 slave.cpp:1891] Status update manager > successfully handled status update TASK_RUNNING (UUID: > 4e914b5c-8a53-4ea0-acea-eea9f451b378) for task 0 of framework > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:04.472378 31448 sched.cpp:395] Scheduler::statusUpdate took > 38.305us > I0826 23:24:04.472564 31395 master.cpp:481] Master terminating > I0826 23:24:04.474488 31395 master.cpp:209] Shutting down master > I0826 23:24:04.474980 31395 master.hpp:329] Removing task 0 with resources > cpus(*):1; mem(*):500 on slave 201308262324-1015726915-53338-31395-0 ( > janus.apache.org) > I0826 23:24:04.472781 31446 slave.cpp:1897] Sending acknowledgement for > status update TASK_RUNNING (UUID: 4e914b5c-8a53-4ea0-acea-eea9f451b378) for > task 0 of framework 201308262324-1015726915-53338-31395-0000 to > executor(40)@67.195.138.60:53338 > I0826 23:24:04.481717 31395 master.cpp:237] Removing slave > 201308262324-1015726915-53338-31395-0 (janus.apache.org) > I0826 23:24:04.481755 31449 hierarchical_allocator_process.hpp:598] > Recovered cpus(*):1; mem(*):500 (total allocatable: cpus(*):2; mem(*):1024; > disk(*):23038; ports(*):[31000-32000]) on slave > 201308262324-1015726915-53338-31395-0 from framework > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:04.482074 31443 exec.cpp:331] Executor received status update > acknowledgement 4e914b5c-8a53-4ea0-acea-eea9f451b378 for task 0 of > framework 201308262324-1015726915-53338-31395-0000 > I0826 23:24:04.482095 31446 slave.cpp:1972] [email protected]:53338exited > W0826 23:24:04.483878 31446 slave.cpp:1975] Master disconnected! Waiting > for a new master to be elected > I0826 23:24:04.482125 31442 status_update_manager.cpp:375] Received status > update acknowledgement (UUID: 4e914b5c-8a53-4ea0-acea-eea9f451b378) for > task 0 of framework 201308262324-1015726915-53338-31395-0000 > I0826 23:24:04.487426 31442 slave.cpp:1387] Status update manager > successfully handled status update acknowledgement (UUID: > 4e914b5c-8a53-4ea0-acea-eea9f451b378) for task 0 of framework > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:04.488179 31449 master.cpp:265] Master started on > 67.195.138.60:53338 > I0826 23:24:04.492727 31449 master.cpp:280] Master ID: > 201308262324-1015726915-53338-31395 > W0826 23:24:04.504068 31443 master.cpp:83] No whitelist given. Advertising > offers for all slaves > I0826 23:24:04.504299 31448 hierarchical_allocator_process.hpp:291] > Initializing hierarchical allocator process with master : > [email protected]:53338 > I0826 23:24:04.540698 31444 detector.cpp:234] Master detector ( > [email protected]:53338) connected to ZooKeeper ... > I0826 23:24:04.540777 31444 detector.cpp:251] Trying to create path > '/znode' in ZooKeeper > I0826 23:24:04.540971 31447 detector.cpp:420] Master detector > (scheduler(113)@67.195.138.60:53338) found 0 registered masters > I0826 23:24:04.541216 31445 detector.cpp:420] Master detector (slave(131)@ > 67.195.138.60:53338) found 0 registered masters > I0826 23:24:04.542078 31445 detector.cpp:441] Master detector (slave(131)@ > 67.195.138.60:53338) couldn't find any masters > I0826 23:24:04.542567 31445 slave.cpp:584] Lost master(s) ... waiting > I0826 23:24:04.541568 31447 detector.cpp:441] Master detector > (scheduler(113)@67.195.138.60:53338) couldn't find any masters > I0826 23:24:04.548354 31447 sched.cpp:202] No master detected, waiting for > another master > I0826 23:24:04.548676 31447 sched.cpp:214] Scheduler::disconnected took > 13.62us > I0826 23:24:04.549679 31444 detector.cpp:281] Created ephemeral/sequence > znode at '/znode/0000000002' > I0826 23:24:04.550272 31443 detector.cpp:420] Master detector > (scheduler(113)@67.195.138.60:53338) found 1 registered masters > I0826 23:24:04.550137 31448 detector.cpp:420] Master detector (slave(131)@ > 67.195.138.60:53338) found 1 registered masters > I0826 23:24:04.550741 31444 detector.cpp:420] Master detector ( > [email protected]:53338) found 1 registered masters > I0826 23:24:04.551309 31443 detector.cpp:467] Master detector > (scheduler(113)@67.195.138.60:53338) got new master pid: > [email protected]:53338 > I0826 23:24:04.552222 31443 sched.cpp:178] New master at > [email protected]:53338 > W0826 23:24:04.552736 31443 master.cpp:717] Ignoring re-register framework > message since not elected yet > I0826 23:24:04.552295 31444 detector.cpp:467] Master detector ( > [email protected]:53338) got new master pid: > [email protected]:53338 > I0826 23:24:04.551817 31448 detector.cpp:467] Master detector (slave(131)@ > 67.195.138.60:53338) got new master pid: [email protected]:53338 > I0826 23:24:04.559463 31444 master.cpp:645] Elected as master! > I0826 23:24:04.560153 31449 slave.cpp:550] New master detected at > [email protected]:53338 > I0826 23:24:04.560919 31447 status_update_manager.cpp:157] New master > detected at [email protected]:53338 > I0826 23:24:05.505043 31446 hierarchical_allocator_process.hpp:687] No > resources available to allocate! > I0826 23:24:05.531718 31446 hierarchical_allocator_process.hpp:649] > Performed allocation for 0 slaves in 26.689654ms > I0826 23:24:05.553195 31443 master.cpp:737] Re-registering framework > 201308262324-1015726915-53338-31395-0000 at scheduler(113)@ > 67.195.138.60:53338 > I0826 23:24:05.553501 31444 sched.cpp:237] Framework registered with > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:05.553683 31442 hierarchical_allocator_process.hpp:321] Added > framework 201308262324-1015726915-53338-31395-0000 > I0826 23:24:05.561470 31448 master.cpp:1162] Attempting to re-register > slave 201308262324-1015726915-53338-31395-0 at slave(131)@ > 67.195.138.60:53338 (janus.apache.org) > I0826 23:24:05.569041 31444 sched.cpp:251] Scheduler::registered took > 19.163us > I0826 23:24:05.569350 31442 hierarchical_allocator_process.hpp:687] No > resources available to allocate! > I0826 23:24:05.570628 31448 master.cpp:2081] Adding slave > 201308262324-1015726915-53338-31395-0 at janus.apache.org with cpus(*):2; > mem(*):1024; disk(*):23038; ports(*):[31000-32000] > I0826 23:24:05.571532 31442 hierarchical_allocator_process.hpp:649] > Performed allocation for 0 slaves in 2.182299ms > I0826 23:24:05.572052 31446 slave.cpp:651] Re-registered with master > [email protected]:53338 > I0826 23:24:05.572103 31448 master.hpp:316] Adding task 0 with resources > cpus(*):1; mem(*):500 on slave 201308262324-1015726915-53338-31395-0 ( > janus.apache.org) > I0826 23:24:05.573590 31447 slave.cpp:1333] Updating framework > 201308262324-1015726915-53338-31395-0000 pid to scheduler(113)@ > 67.195.138.60:53338 > I0826 23:24:05.573768 31449 hierarchical_allocator_process.hpp:434] Added > slave 201308262324-1015726915-53338-31395-0 (janus.apache.org) with > cpus(*):2; mem(*):1024; disk(*):23038; ports(*):[31000-32000] (and > cpus(*):1; mem(*):524; disk(*):23038; ports(*):[31000-32000] available) > I0826 23:24:05.574491 31449 hierarchical_allocator_process.hpp:712] > Offering cpus(*):1; mem(*):524; disk(*):23038; ports(*):[31000-32000] on > slave 201308262324-1015726915-53338-31395-0 to framework > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:05.575062 31449 hierarchical_allocator_process.hpp:669] > Performed allocation for slave 201308262324-1015726915-53338-31395-0 in > 612.764us > I0826 23:24:05.575139 31446 master.hpp:339] Adding offer > 201308262324-1015726915-53338-31395-0 with resources cpus(*):1; mem(*):524; > disk(*):23038; ports(*):[31000-32000] on slave > 201308262324-1015726915-53338-31395-0 (janus.apache.org) > I0826 23:24:05.575934 31446 master.cpp:1442] Sending 1 offers to framework > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:05.576522 31444 sched.cpp:340] Scheduler::resourceOffers took > 32.888us > I0826 23:24:05.576828 31395 master.cpp:481] Master terminating > I0826 23:24:05.576906 31444 sched.cpp:498] Stopping framework > '201308262324-1015726915-53338-31395-0000' > I0826 23:24:05.577397 31395 master.cpp:209] Shutting down master > I0826 23:24:05.577410 31443 slave.cpp:1972] [email protected]:53338exited > I0826 23:24:05.580099 31395 master.hpp:329] Removing task 0 with resources > cpus(*):1; mem(*):500 on slave 201308262324-1015726915-53338-31395-0 ( > janus.apache.org) > W0826 23:24:05.594966 31443 slave.cpp:1975] Master disconnected! Waiting > for a new master to be elected > I0826 23:24:05.595598 31395 master.hpp:349] Removing offer > 201308262324-1015726915-53338-31395-0 with resources cpus(*):1; mem(*):524; > disk(*):23038; ports(*):[31000-32000] on slave > 201308262324-1015726915-53338-31395-0 (janus.apache.org) > I0826 23:24:05.595624 31442 hierarchical_allocator_process.hpp:598] > Recovered cpus(*):1; mem(*):500 (total allocatable: cpus(*):1; mem(*):500) > on slave 201308262324-1015726915-53338-31395-0 from framework > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:05.596436 31395 master.cpp:237] Removing slave > 201308262324-1015726915-53338-31395-0 (janus.apache.org) > I0826 23:24:05.598697 31443 slave.cpp:460] Slave terminating > I0826 23:24:05.598764 31443 slave.cpp:1168] Asked to shut down framework > 201308262324-1015726915-53338-31395-0000 by @0.0.0.0:0 > I0826 23:24:05.599071 31443 slave.cpp:1193] Shutting down framework > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:05.599550 31443 slave.cpp:2423] Shutting down executor > 'default' of framework 201308262324-1015726915-53338-31395-0000 > I0826 23:24:05.600046 31447 exec.cpp:371] Executor asked to shutdown > > GMOCK WARNING: > Uninteresting mock function call - returning directly. > Function call: shutdown(0x2b1c880190c0) > Stack trace: > I0826 23:24:05.600539 31447 exec.cpp:386] Executor::shutdown took 26.437us > [ OK ] AllocatorZooKeeperTest/0.FrameworkReregistersFirst (2284 ms) > [ RUN ] AllocatorZooKeeperTest/0.SlaveReregistersFirst > I0826 23:24:05.601713 31442 master.cpp:265] Master started on > 67.195.138.60:53338 > I0826 23:24:05.601804 31442 master.cpp:280] Master ID: > 201308262324-1015726915-53338-31395 > I0826 23:24:05.602412 31447 hierarchical_allocator_process.hpp:291] > Initializing hierarchical allocator process with master : > [email protected]:53338 > I0826 23:24:05.602613 31445 slave.cpp:114] Slave started on 132)@ > 67.195.138.60:53338 > W0826 23:24:05.602578 31444 master.cpp:83] No whitelist given. Advertising > offers for all slaves > I0826 23:24:05.603168 31445 slave.cpp:214] Slave resources: cpus(*):2; > mem(*):1024; disk(*):23038; ports(*):[31000-32000] > I0826 23:24:05.614043 31448 slave.cpp:405] Finished recovery > I0826 23:24:05.625529 31444 detector.cpp:234] Master detector ( > [email protected]:53338) connected to ZooKeeper ... > I0826 23:24:05.625736 31444 detector.cpp:251] Trying to create path > '/znode' in ZooKeeper > I0826 23:24:05.626380 31442 detector.cpp:234] Master detector (slave(132)@ > 67.195.138.60:53338) connected to ZooKeeper ... > I0826 23:24:05.626509 31442 detector.cpp:251] Trying to create path > '/znode' in ZooKeeper > I0826 23:24:05.627017 31449 detector.cpp:234] Master detector > (scheduler(114)@67.195.138.60:53338) connected to ZooKeeper ... > I0826 23:24:05.627444 31449 detector.cpp:251] Trying to create path > '/znode' in ZooKeeper > I0826 23:24:05.640588 31444 detector.cpp:281] Created ephemeral/sequence > znode at '/znode/0000000004' > I0826 23:24:05.640866 31442 detector.cpp:420] Master detector (slave(132)@ > 67.195.138.60:53338) found 1 registered masters > I0826 23:24:05.641338 31444 detector.cpp:420] Master detector ( > [email protected]:53338) found 1 registered masters > I0826 23:24:05.641867 31442 detector.cpp:467] Master detector (slave(132)@ > 67.195.138.60:53338) got new master pid: [email protected]:53338 > I0826 23:24:05.642060 31442 slave.cpp:550] New master detected at > [email protected]:53338 > I0826 23:24:05.642117 31449 detector.cpp:420] Master detector > (scheduler(114)@67.195.138.60:53338) found 1 registered masters > I0826 23:24:05.642241 31444 detector.cpp:467] Master detector ( > [email protected]:53338) got new master pid: > [email protected]:53338 > I0826 23:24:05.642505 31447 status_update_manager.cpp:157] New master > detected at [email protected]:53338 > W0826 23:24:05.642565 31448 master.cpp:1037] Ignoring register slave > message from janus.apache.org since not elected yet > I0826 23:24:05.643558 31449 detector.cpp:467] Master detector > (scheduler(114)@67.195.138.60:53338) got new master pid: > [email protected]:53338 > I0826 23:24:05.644429 31448 master.cpp:645] Elected as master! > I0826 23:24:05.644984 31449 sched.cpp:178] New master at > [email protected]:53338 > I0826 23:24:05.646739 31442 master.cpp:695] Registering framework > 201308262324-1015726915-53338-31395-0000 at scheduler(114)@ > 67.195.138.60:53338 > I0826 23:24:05.647022 31449 sched.cpp:237] Framework registered with > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:05.647102 31446 hierarchical_allocator_process.hpp:321] Added > framework 201308262324-1015726915-53338-31395-0000 > I0826 23:24:05.647703 31449 sched.cpp:251] Scheduler::registered took > 17.747us > I0826 23:24:05.647954 31446 hierarchical_allocator_process.hpp:687] No > resources available to allocate! > I0826 23:24:05.649072 31446 hierarchical_allocator_process.hpp:649] > Performed allocation for 0 slaves in 1.120037ms > I0826 23:24:06.603658 31443 hierarchical_allocator_process.hpp:687] No > resources available to allocate! > I0826 23:24:06.630934 31443 hierarchical_allocator_process.hpp:649] > Performed allocation for 0 slaves in 27.29318ms > I0826 23:24:06.642896 31443 master.cpp:1068] Attempting to register slave > on janus.apache.org at slave(132)@67.195.138.60:53338 > I0826 23:24:06.643010 31443 master.cpp:2081] Adding slave > 201308262324-1015726915-53338-31395-0 at janus.apache.org with cpus(*):2; > mem(*):1024; disk(*):23038; ports(*):[31000-32000] > I0826 23:24:06.667451 31445 slave.cpp:610] Registered with master > [email protected]:53338; given slave ID > 201308262324-1015726915-53338-31395-0 > I0826 23:24:06.667529 31444 hierarchical_allocator_process.hpp:434] Added > slave 201308262324-1015726915-53338-31395-0 (janus.apache.org) with > cpus(*):2; mem(*):1024; disk(*):23038; ports(*):[31000-32000] (and > cpus(*):2; mem(*):1024; disk(*):23038; ports(*):[31000-32000] available) > I0826 23:24:06.668153 31444 hierarchical_allocator_process.hpp:712] > Offering cpus(*):2; mem(*):1024; disk(*):23038; ports(*):[31000-32000] on > slave 201308262324-1015726915-53338-31395-0 to framework > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:06.668721 31444 hierarchical_allocator_process.hpp:669] > Performed allocation for slave 201308262324-1015726915-53338-31395-0 in > 606.428us > I0826 23:24:06.668792 31445 master.hpp:339] Adding offer > 201308262324-1015726915-53338-31395-0 with resources cpus(*):2; > mem(*):1024; disk(*):23038; ports(*):[31000-32000] on slave > 201308262324-1015726915-53338-31395-0 (janus.apache.org) > I0826 23:24:06.670524 31445 master.cpp:1442] Sending 1 offers to framework > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:06.671212 31445 sched.cpp:340] Scheduler::resourceOffers took > 206.558us > I0826 23:24:06.671491 31445 master.cpp:1679] Processing reply for offer > 201308262324-1015726915-53338-31395-0 on slave > 201308262324-1015726915-53338-31395-0 (janus.apache.org) for framework > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:06.671983 31445 master.hpp:316] Adding task 0 with resources > cpus(*):1; mem(*):500 on slave 201308262324-1015726915-53338-31395-0 ( > janus.apache.org) > I0826 23:24:06.672392 31445 master.cpp:1799] Launching task 0 of framework > 201308262324-1015726915-53338-31395-0000 with resources cpus(*):1; > mem(*):500 on slave 201308262324-1015726915-53338-31395-0 ( > janus.apache.org) > I0826 23:24:06.678482 31447 slave.cpp:773] Got assigned task 0 for > framework 201308262324-1015726915-53338-31395-0000 > I0826 23:24:06.678956 31447 slave.cpp:884] Launching task 0 for framework > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:06.678577 31446 hierarchical_allocator_process.hpp:508] > Framework 201308262324-1015726915-53338-31395-0000 left cpus(*):1; > mem(*):524; disk(*):23038; ports(*):[31000-32000] unused on slave > 201308262324-1015726915-53338-31395-0 > I0826 23:24:06.678561 31445 master.hpp:349] Removing offer > 201308262324-1015726915-53338-31395-0 with resources cpus(*):2; > mem(*):1024; disk(*):23038; ports(*):[31000-32000] on slave > 201308262324-1015726915-53338-31395-0 (janus.apache.org) > I0826 23:24:06.679827 31446 hierarchical_allocator_process.hpp:551] > Framework 201308262324-1015726915-53338-31395-0000 filtered slave > 201308262324-1015726915-53338-31395-0 for 5secs > I0826 23:24:06.681046 31447 paths.hpp:336] Created executor directory > '/tmp/AllocatorZooKeeperTest_0_SlaveReregistersFirst_fod9nv/slaves/201308262324-1015726915-53338-31395-0/frameworks/201308262324-1015726915-53338-31395-0000/executors/default/runs/479bd3b5-72a5-412f-b165-7c824152f076' > I0826 23:24:06.682035 31447 slave.cpp:995] Queuing task '0' for executor > default of framework '201308262324-1015726915-53338-31395-0000 > I0826 23:24:06.682441 31447 slave.cpp:532] Successfully attached file > '/tmp/AllocatorZooKeeperTest_0_SlaveReregistersFirst_fod9nv/slaves/201308262324-1015726915-53338-31395-0/frameworks/201308262324-1015726915-53338-31395-0000/executors/default/runs/479bd3b5-72a5-412f-b165-7c824152f076' > I0826 23:24:06.683823 31442 exec.cpp:174] Executor started at: > executor(41)@67.195.138.60:53338 with pid 31395 > I0826 23:24:06.689611 31442 slave.cpp:1441] Got registration for executor > 'default' of framework 201308262324-1015726915-53338-31395-0000 > I0826 23:24:06.690002 31442 slave.cpp:1562] Flushing queued task 0 for > executor 'default' of framework 201308262324-1015726915-53338-31395-0000 > I0826 23:24:06.690055 31448 exec.cpp:198] Executor registered on slave > 201308262324-1015726915-53338-31395-0 > I0826 23:24:06.692275 31448 exec.cpp:210] Executor::registered took > 12.582us > I0826 23:24:06.692371 31448 exec.cpp:285] Executor asked to run task '0' > I0826 23:24:06.692738 31448 exec.cpp:294] Executor::launchTask took > 22.403us > I0826 23:24:06.694619 31448 exec.cpp:501] Executor sending status update > TASK_RUNNING (UUID: 1bf92f2e-61ae-4e3f-aa30-66b3ddea509e) for task 0 of > framework 201308262324-1015726915-53338-31395-0000 > I0826 23:24:06.694809 31449 slave.cpp:1772] Handling status update > TASK_RUNNING (UUID: 1bf92f2e-61ae-4e3f-aa30-66b3ddea509e) for task 0 of > framework 201308262324-1015726915-53338-31395-0000 from executor(41)@ > 67.195.138.60:53338 > I0826 23:24:06.700773 31449 status_update_manager.cpp:300] Received status > update TASK_RUNNING (UUID: 1bf92f2e-61ae-4e3f-aa30-66b3ddea509e) for task 0 > of framework 201308262324-1015726915-53338-31395-0000 > I0826 23:24:06.701041 31449 status_update_manager.cpp:471] Creating > StatusUpdate stream for task 0 of framework > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:06.701524 31449 status_update_manager.cpp:351] Forwarding > status update TASK_RUNNING (UUID: 1bf92f2e-61ae-4e3f-aa30-66b3ddea509e) for > task 0 of framework 201308262324-1015726915-53338-31395-0000 to > [email protected]:53338 > I0826 23:24:06.702049 31449 master.cpp:1208] Status update TASK_RUNNING > (UUID: 1bf92f2e-61ae-4e3f-aa30-66b3ddea509e) for task 0 of framework > 201308262324-1015726915-53338-31395-0000 from slave(132)@ > 67.195.138.60:53338 > I0826 23:24:06.702751 31449 sched.cpp:395] Scheduler::statusUpdate took > 16.166us > I0826 23:24:06.702915 31395 master.cpp:481] Master terminating > I0826 23:24:06.702111 31443 slave.cpp:1891] Status update manager > successfully handled status update TASK_RUNNING (UUID: > 1bf92f2e-61ae-4e3f-aa30-66b3ddea509e) for task 0 of framework > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:06.704164 31443 slave.cpp:1897] Sending acknowledgement for > status update TASK_RUNNING (UUID: 1bf92f2e-61ae-4e3f-aa30-66b3ddea509e) for > task 0 of framework 201308262324-1015726915-53338-31395-0000 to > executor(41)@67.195.138.60:53338 > I0826 23:24:06.704736 31446 exec.cpp:331] Executor received status update > acknowledgement 1bf92f2e-61ae-4e3f-aa30-66b3ddea509e for task 0 of > framework 201308262324-1015726915-53338-31395-0000 > I0826 23:24:06.704742 31443 slave.cpp:1972] [email protected]:53338exited > W0826 23:24:06.711784 31443 slave.cpp:1975] Master disconnected! Waiting > for a new master to be elected > I0826 23:24:06.704812 31445 status_update_manager.cpp:375] Received status > update acknowledgement (UUID: 1bf92f2e-61ae-4e3f-aa30-66b3ddea509e) for > task 0 of framework 201308262324-1015726915-53338-31395-0000 > I0826 23:24:06.703708 31395 master.cpp:209] Shutting down master > I0826 23:24:06.712635 31444 slave.cpp:1387] Status update manager > successfully handled status update acknowledgement (UUID: > 1bf92f2e-61ae-4e3f-aa30-66b3ddea509e) for task 0 of framework > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:06.713047 31395 master.hpp:329] Removing task 0 with resources > cpus(*):1; mem(*):500 on slave 201308262324-1015726915-53338-31395-0 ( > janus.apache.org) > I0826 23:24:06.714067 31395 master.cpp:237] Removing slave > 201308262324-1015726915-53338-31395-0 (janus.apache.org) > I0826 23:24:06.714121 31449 hierarchical_allocator_process.hpp:598] > Recovered cpus(*):1; mem(*):500 (total allocatable: cpus(*):2; mem(*):1024; > disk(*):23038; ports(*):[31000-32000]) on slave > 201308262324-1015726915-53338-31395-0 from framework > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:06.716498 31444 master.cpp:265] Master started on > 67.195.138.60:53338 > I0826 23:24:06.716603 31444 master.cpp:280] Master ID: > 201308262324-1015726915-53338-31395 > W0826 23:24:06.723125 31442 master.cpp:83] No whitelist given. Advertising > offers for all slaves > I0826 23:24:06.723356 31449 hierarchical_allocator_process.hpp:291] > Initializing hierarchical allocator process with master : > [email protected]:53338 > I0826 23:24:06.765470 31448 detector.cpp:234] Master detector ( > [email protected]:53338) connected to ZooKeeper ... > I0826 23:24:06.765573 31448 detector.cpp:251] Trying to create path > '/znode' in ZooKeeper > I0826 23:24:06.765765 31446 detector.cpp:420] Master detector > (scheduler(114)@67.195.138.60:53338) found 0 registered masters > I0826 23:24:06.766008 31444 detector.cpp:420] Master detector (slave(132)@ > 67.195.138.60:53338) found 0 registered masters > I0826 23:24:06.766377 31446 detector.cpp:441] Master detector > (scheduler(114)@67.195.138.60:53338) couldn't find any masters > I0826 23:24:06.766842 31444 detector.cpp:441] Master detector (slave(132)@ > 67.195.138.60:53338) couldn't find any masters > I0826 23:24:06.767403 31446 sched.cpp:202] No master detected, waiting for > another master > I0826 23:24:06.778527 31446 sched.cpp:214] Scheduler::disconnected took > 10.469us > I0826 23:24:06.767835 31444 slave.cpp:584] Lost master(s) ... waiting > I0826 23:24:06.779032 31448 detector.cpp:281] Created ephemeral/sequence > znode at '/znode/0000000006' > I0826 23:24:06.779368 31447 detector.cpp:420] Master detector > (scheduler(114)@67.195.138.60:53338) found 1 registered masters > I0826 23:24:06.779600 31445 detector.cpp:420] Master detector (slave(132)@ > 67.195.138.60:53338) found 1 registered masters > I0826 23:24:06.780794 31448 detector.cpp:420] Master detector ( > [email protected]:53338) found 1 registered masters > I0826 23:24:06.780885 31447 detector.cpp:467] Master detector > (scheduler(114)@67.195.138.60:53338) got new master pid: > [email protected]:53338 > I0826 23:24:06.781839 31447 sched.cpp:178] New master at > [email protected]:53338 > I0826 23:24:06.781371 31445 detector.cpp:467] Master detector (slave(132)@ > 67.195.138.60:53338) got new master pid: [email protected]:53338 > I0826 23:24:06.782780 31445 slave.cpp:550] New master detected at > [email protected]:53338 > I0826 23:24:06.783283 31445 status_update_manager.cpp:157] New master > detected at [email protected]:53338 > W0826 23:24:06.783366 31447 master.cpp:1095] Ignoring re-register slave > message from janus.apache.org since not elected yet > I0826 23:24:06.781901 31448 detector.cpp:467] Master detector ( > [email protected]:53338) got new master pid: > [email protected]:53338 > I0826 23:24:06.784698 31448 master.cpp:645] Elected as master! > I0826 23:24:07.643393 31449 master.cpp:1162] Attempting to re-register > slave 201308262324-1015726915-53338-31395-0 at slave(132)@ > 67.195.138.60:53338 (janus.apache.org) > I0826 23:24:07.663753 31449 master.cpp:2081] Adding slave > 201308262324-1015726915-53338-31395-0 at janus.apache.org with cpus(*):2; > mem(*):1024; disk(*):23038; ports(*):[31000-32000] > I0826 23:24:07.664268 31443 slave.cpp:651] Re-registered with master > [email protected]:53338 > I0826 23:24:07.664331 31449 master.hpp:316] Adding task 0 with resources > cpus(*):1; mem(*):500 on slave 201308262324-1015726915-53338-31395-0 ( > janus.apache.org) > W0826 23:24:07.699620 31449 master.cpp:2173] Possibly orphaned task 0 of > framework 201308262324-1015726915-53338-31395-0000 running on slave > 201308262324-1015726915-53338-31395-0 (janus.apache.org) > I0826 23:24:07.700306 31443 hierarchical_allocator_process.hpp:434] Added > slave 201308262324-1015726915-53338-31395-0 (janus.apache.org) with > cpus(*):2; mem(*):1024; disk(*):23038; ports(*):[31000-32000] (and > cpus(*):1; mem(*):524; disk(*):23038; ports(*):[31000-32000] available) > I0826 23:24:07.700558 31443 hierarchical_allocator_process.hpp:669] > Performed allocation for slave 201308262324-1015726915-53338-31395-0 in > 14.432us > I0826 23:24:07.735359 31444 hierarchical_allocator_process.hpp:649] > Performed allocation for 1 slaves in 9.663us > I0826 23:24:07.782487 31445 master.cpp:737] Re-registering framework > 201308262324-1015726915-53338-31395-0000 at scheduler(114)@ > 67.195.138.60:53338 > I0826 23:24:07.782719 31448 sched.cpp:237] Framework registered with > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:07.782776 31445 slave.cpp:1333] Updating framework > 201308262324-1015726915-53338-31395-0000 pid to scheduler(114)@ > 67.195.138.60:53338 > I0826 23:24:07.782887 31446 hierarchical_allocator_process.hpp:321] Added > framework 201308262324-1015726915-53338-31395-0000 > I0826 23:24:07.782979 31448 sched.cpp:251] Scheduler::registered took > 12.17us > I0826 23:24:07.783866 31446 hierarchical_allocator_process.hpp:712] > Offering cpus(*):1; mem(*):524; disk(*):23038; ports(*):[31000-32000] on > slave 201308262324-1015726915-53338-31395-0 to framework > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:07.784921 31446 hierarchical_allocator_process.hpp:649] > Performed allocation for 1 slaves in 1.097745ms > I0826 23:24:07.785051 31442 master.hpp:339] Adding offer > 201308262324-1015726915-53338-31395-0 with resources cpus(*):1; mem(*):524; > disk(*):23038; ports(*):[31000-32000] on slave > 201308262324-1015726915-53338-31395-0 (janus.apache.org) > I0826 23:24:07.785779 31442 master.cpp:1442] Sending 1 offers to framework > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:07.786321 31449 sched.cpp:340] Scheduler::resourceOffers took > 22.841us > I0826 23:24:07.786504 31395 master.cpp:481] Master terminating > I0826 23:24:07.786717 31449 sched.cpp:498] Stopping framework > '201308262324-1015726915-53338-31395-0000' > I0826 23:24:07.787204 31395 master.cpp:209] Shutting down master > I0826 23:24:07.787210 31443 slave.cpp:1972] [email protected]:53338exited > I0826 23:24:07.788187 31395 master.hpp:329] Removing task 0 with resources > cpus(*):1; mem(*):500 on slave 201308262324-1015726915-53338-31395-0 ( > janus.apache.org) > W0826 23:24:07.788645 31443 slave.cpp:1975] Master disconnected! Waiting > for a new master to be elected > I0826 23:24:07.789242 31445 hierarchical_allocator_process.hpp:598] > Recovered cpus(*):1; mem(*):500 (total allocatable: cpus(*):1; mem(*):500) > on slave 201308262324-1015726915-53338-31395-0 from framework > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:07.789288 31395 master.hpp:349] Removing offer > 201308262324-1015726915-53338-31395-0 with resources cpus(*):1; mem(*):524; > disk(*):23038; ports(*):[31000-32000] on slave > 201308262324-1015726915-53338-31395-0 (janus.apache.org) > I0826 23:24:07.800119 31395 master.cpp:237] Removing slave > 201308262324-1015726915-53338-31395-0 (janus.apache.org) > I0826 23:24:07.800989 31395 slave.cpp:460] Slave terminating > I0826 23:24:07.801095 31395 slave.cpp:1168] Asked to shut down framework > 201308262324-1015726915-53338-31395-0000 by @0.0.0.0:0 > I0826 23:24:07.801498 31395 slave.cpp:1193] Shutting down framework > 201308262324-1015726915-53338-31395-0000 > I0826 23:24:07.801982 31395 slave.cpp:2423] Shutting down executor > 'default' of framework 201308262324-1015726915-53338-31395-0000 > I0826 23:24:07.810938 31448 exec.cpp:371] Executor asked to shutdown > > GMOCK WARNING: > Uninteresting mock function call - returning directly. > Function call: shutdown(0x2b1c9802d7f0) > Stack trace: > I0826 23:24:07.811175 31448 exec.cpp:386] Executor::shutdown took 21.717us > [ OK ] AllocatorZooKeeperTest/0.SlaveReregistersFirst (2210 ms) > I0826 23:24:07.814152 31395 zookeeper_test_server.cpp:93] Shutdown > ZooKeeperTestServer on port 47107 > [----------] 2 tests from AllocatorZooKeeperTest/0 (4494 ms total) > > [----------] Global test environment tear-down > [==========] 220 tests from 39 test cases ran. (155381 ms total) > [ PASSED ] 219 tests. > [ FAILED ] 1 test, listed below: > [ FAILED ] SlaveRecoveryTest/0.MultipleFrameworks, where TypeParam = > mesos::internal::slave::ProcessIsolator > > 1 FAILED TEST > make[3]: *** [check-local] Error 1 > make[3]: Leaving directory `< > https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-In-Src-Set-JAVA_HOME/ws/src > '> > make[2]: *** [check-am] Error 2 > make[2]: Leaving directory `< > https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-In-Src-Set-JAVA_HOME/ws/src > '> > make[1]: *** [check] Error 2 > make[1]: Leaving directory `< > https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-In-Src-Set-JAVA_HOME/ws/src > '> > make: *** [check-recursive] Error 1 > Build step 'Execute shell' marked build as failure >
