See <https://builds.apache.org/job/Mesos-Buildbot/BUILDTOOL=cmake,COMPILER=gcc,CONFIGURATION=--verbose%20--disable-libtool-wrappers,ENVIRONMENT=GLOG_v=1%20MESOS_VERBOSE=1,OS=ubuntu%3A14.04,label_exp=(ubuntu)&&(!ubuntu-us1)&&(!ubuntu-eu2)&&(!qnode3)&&(!H23)/4447/display/redirect?page=changes>
Changes: [alexr] Expanded description of the --recovery_timeout flag. ------------------------------------------ [...truncated 20.63 MB...] 3: I1113 11:04:56.049720 17341 sched.cpp:765] Scheduler::registered took 33849ns 3: I1113 11:04:56.049829 17333 slave.cpp:6422] Finished recovery 3: I1113 11:04:56.049974 17321 hierarchical.cpp:1450] Performed allocation for 0 agents in 83655ns 3: I1113 11:04:56.050839 17340 status_update_manager.cpp:177] Pausing sending status updates 3: I1113 11:04:56.050876 17339 slave.cpp:1007] New master detected at [email protected]:44093 3: I1113 11:04:56.050954 17339 slave.cpp:1042] Detecting new master 3: I1113 11:04:56.053962 17329 hierarchical.cpp:1450] Performed allocation for 0 agents in 95032ns 3: I1113 11:04:56.054976 17328 slave.cpp:1069] Authenticating with master [email protected]:44093 3: I1113 11:04:56.055078 17328 slave.cpp:1078] Using default CRAM-MD5 authenticatee 3: I1113 11:04:56.055397 17331 authenticatee.cpp:121] Creating new client SASL connection 3: I1113 11:04:56.055811 17323 master.cpp:8286] Authenticating slave(540)@172.17.0.2:44093 3: I1113 11:04:56.055968 17343 authenticator.cpp:414] Starting authentication session for crammd5-authenticatee(1087)@172.17.0.2:44093 3: I1113 11:04:56.056298 17330 authenticator.cpp:98] Creating new server SASL connection 3: I1113 11:04:56.056645 17337 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5 3: I1113 11:04:56.056677 17337 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5' 3: I1113 11:04:56.056855 17332 authenticator.cpp:204] Received SASL authentication start 3: I1113 11:04:56.056946 17332 authenticator.cpp:326] Authentication requires more steps 3: I1113 11:04:56.057128 17324 authenticatee.cpp:259] Received SASL authentication step 3: I1113 11:04:56.057322 17338 authenticator.cpp:232] Received SASL authentication step 3: I1113 11:04:56.057366 17338 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: '08347a511b13' server FQDN: '08347a511b13' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false 3: I1113 11:04:56.057379 17338 auxprop.cpp:181] Looking up auxiliary property '*userPassword' 3: I1113 11:04:56.057416 17338 auxprop.cpp:181] Looking up auxiliary property '*cmusaslsecretCRAM-MD5' 3: I1113 11:04:56.057440 17338 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: '08347a511b13' server FQDN: '08347a511b13' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true 3: I1113 11:04:56.057454 17338 auxprop.cpp:131] Skipping auxiliary property '*userPassword' since SASL_AUXPROP_AUTHZID == true 3: I1113 11:04:56.057462 17338 auxprop.cpp:131] Skipping auxiliary property '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true 3: I1113 11:04:56.057482 17338 authenticator.cpp:318] Authentication success 3: I1113 11:04:56.057605 17336 authenticatee.cpp:299] Authentication success 3: I1113 11:04:56.057708 17325 master.cpp:8316] Successfully authenticated principal 'test-principal' at slave(540)@172.17.0.2:44093 3: I1113 11:04:56.057749 17338 authenticator.cpp:432] Authentication session cleanup for crammd5-authenticatee(1087)@172.17.0.2:44093 3: I1113 11:04:56.058046 17344 slave.cpp:1161] Successfully authenticated with master [email protected]:44093 3: I1113 11:04:56.058331 17344 slave.cpp:1682] Will retry registration in 17.819354ms if necessary 3: I1113 11:04:56.058580 17321 master.cpp:6033] Received register agent message from slave(540)@172.17.0.2:44093 (08347a511b13) 3: I1113 11:04:56.058751 17321 master.cpp:3871] Authorizing agent with principal 'test-principal' 3: I1113 11:04:56.059320 17334 master.cpp:6093] Authorized registration of agent at slave(540)@172.17.0.2:44093 (08347a511b13) 3: I1113 11:04:56.059469 17334 master.cpp:6186] Registering agent at slave(540)@172.17.0.2:44093 (08347a511b13) with id 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 3: I1113 11:04:56.060032 17322 registrar.cpp:495] Applied 1 operations in 83930ns; attempting to update the registry 3: I1113 11:04:56.060570 17340 hierarchical.cpp:1450] Performed allocation for 0 agents in 146054ns 3: I1113 11:04:56.060966 17322 registrar.cpp:552] Successfully updated the registry in 777984ns 3: I1113 11:04:56.061264 17331 master.cpp:6233] Admitted agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 at slave(540)@172.17.0.2:44093 (08347a511b13) 3: I1113 11:04:56.062137 17343 slave.cpp:1207] Registered with master [email protected]:44093; given agent ID 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 3: I1113 11:04:56.062026 17331 master.cpp:6264] Registered agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 at slave(540)@172.17.0.2:44093 (08347a511b13) with [{"name":"cpus","scalar":{"value":1.0},"type":"SCALAR"},{"name":"mem","scalar":{"value":512.0},"type":"SCALAR"},{"name":"disk","scalar":{"value":367463.0},"type":"SCALAR"},{"name":"ports","ranges":{"range":[{"begin":31000,"end":32000}]},"type":"RANGES"}] 3: I1113 11:04:56.062321 17332 status_update_manager.cpp:184] Resuming sending status updates 3: I1113 11:04:56.062454 17330 hierarchical.cpp:593] Added agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 (08347a511b13) with cpus:1; mem:512; disk:367463; ports:[31000-32000] (allocated: {}) 3: I1113 11:04:56.062539 17343 slave.cpp:1227] Checkpointing SlaveInfo to '/tmp/ReservationTest_PreventUnreservingAlienResources_sM4Wb0/meta/slaves/5f212a45-c32c-48f2-8392-e37909bcc50b-S0/slave.info' 3: I1113 11:04:56.063073 17343 slave.cpp:1295] Forwarding total oversubscribed resources {} 3: I1113 11:04:56.063333 17343 master.cpp:7065] Received update of agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 at slave(540)@172.17.0.2:44093 (08347a511b13) with total oversubscribed resources {} 3: I1113 11:04:56.063560 17343 master.cpp:7083] Ignoring update on agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 at slave(540)@172.17.0.2:44093 (08347a511b13) as it reports no changes 3: I1113 11:04:56.064016 17330 hierarchical.cpp:1450] Performed allocation for 1 agents in 1.354207ms 3: I1113 11:04:56.064736 17326 master.cpp:8116] Sending 1 offers to framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 (framework1) at [email protected]:44093 3: I1113 11:04:56.065477 17336 sched.cpp:921] Scheduler::resourceOffers took 159414ns 3: I1113 11:04:56.066524 17338 hierarchical.cpp:1450] Performed allocation for 1 agents in 197536ns 3: I1113 11:04:56.067996 17333 master.cpp:10016] Removing offer 5f212a45-c32c-48f2-8392-e37909bcc50b-O0 3: I1113 11:04:56.068162 17333 master.cpp:4229] Processing ACCEPT call for offers: [ 5f212a45-c32c-48f2-8392-e37909bcc50b-O0 ] on agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 at slave(540)@172.17.0.2:44093 (08347a511b13) for framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 (framework1) at [email protected]:44093 3: I1113 11:04:56.068363 17333 master.cpp:3656] Authorizing principal 'test-principal' to reserve resources '[{"allocation_info":{"role":"role1"},"name":"mem","reservations":[{"principal":"test-principal","role":"role1","type":"DYNAMIC"}],"scalar":{"value":256.0},"type":"SCALAR"}]' 3: I1113 11:04:56.070134 17340 master.cpp:4562] Applying RESERVE operation for resources [{"allocation_info":{"role":"role1"},"name":"mem","reservations":[{"principal":"test-principal","role":"role1","type":"DYNAMIC"}],"scalar":{"value":256.0},"type":"SCALAR"}] from framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 (framework1) at [email protected]:44093 to agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 at slave(540)@172.17.0.2:44093 (08347a511b13) 3: I1113 11:04:56.070775 17340 master.cpp:9961] Sending updated checkpointed resources mem(reservations: [(DYNAMIC,role1,test-principal)]):256 to agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 at slave(540)@172.17.0.2:44093 (08347a511b13) 3: I1113 11:04:56.071862 17323 slave.cpp:3587] Updated checkpointed resources from {} to mem(reservations: [(DYNAMIC,role1,test-principal)]):256 3: I1113 11:04:56.074899 17337 hierarchical.cpp:849] Updated allocation of framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 on agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 from cpus(allocated: role1):1; mem(allocated: role1):512; disk(allocated: role1):367463; ports(allocated: role1):[31000-32000] to cpus(allocated: role1):1; mem(allocated: role1):256; disk(allocated: role1):367463; ports(allocated: role1):[31000-32000]; mem(allocated: role1)(reservations: [(DYNAMIC,role1,test-principal)]):256 3: I1113 11:04:56.076831 17337 hierarchical.cpp:1125] Recovered cpus(allocated: role1):1; mem(allocated: role1):256; disk(allocated: role1):367463; ports(allocated: role1):[31000-32000]; mem(allocated: role1)(reservations: [(DYNAMIC,role1,test-principal)]):256 (total: cpus:1; mem:256; disk:367463; ports:[31000-32000]; mem(reservations: [(DYNAMIC,role1,test-principal)]):256, allocated: {}) on agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 from framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 3: I1113 11:04:56.078974 17337 hierarchical.cpp:1450] Performed allocation for 1 agents in 1.805156ms 3: I1113 11:04:56.079792 17331 master.cpp:8116] Sending 1 offers to framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 (framework1) at [email protected]:44093 3: I1113 11:04:56.080495 17332 sched.cpp:921] Scheduler::resourceOffers took 178243ns 3: I1113 11:04:56.081637 17341 master.cpp:5433] Processing DECLINE call for offers: [ 5f212a45-c32c-48f2-8392-e37909bcc50b-O1 ] for framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 (framework1) at [email protected]:44093 3: I1113 11:04:56.081943 17320 sched.cpp:232] Version: 1.5.0 3: I1113 11:04:56.082598 17344 sched.cpp:336] New master detected at [email protected]:44093 3: I1113 11:04:56.082633 17341 master.cpp:10016] Removing offer 5f212a45-c32c-48f2-8392-e37909bcc50b-O1 3: I1113 11:04:56.082720 17344 sched.cpp:396] Authenticating with master [email protected]:44093 3: I1113 11:04:56.082741 17344 sched.cpp:403] Using default CRAM-MD5 authenticatee 3: I1113 11:04:56.083037 17327 authenticatee.cpp:121] Creating new client SASL connection 3: I1113 11:04:56.083360 17329 master.cpp:8286] Authenticating [email protected]:44093 3: I1113 11:04:56.083520 17333 authenticator.cpp:414] Starting authentication session for crammd5-authenticatee(1088)@172.17.0.2:44093 3: I1113 11:04:56.083890 17328 authenticator.cpp:98] Creating new server SASL connection 3: I1113 11:04:56.084033 17336 hierarchical.cpp:1125] Recovered cpus(allocated: role1):1; mem(allocated: role1):256; disk(allocated: role1):367463; ports(allocated: role1):[31000-32000]; mem(allocated: role1)(reservations: [(DYNAMIC,role1,test-principal)]):256 (total: cpus:1; mem:256; disk:367463; ports:[31000-32000]; mem(reservations: [(DYNAMIC,role1,test-principal)]):256, allocated: {}) on agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 from framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 3: W1113 11:04:56.084133 17336 hierarchical.cpp:1146] Using 365 days to create the refused resources offer filter because the input value is too big 3: I1113 11:04:56.084156 17336 hierarchical.cpp:1171] Framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 filtered agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 for 365days 3: I1113 11:04:56.084239 17322 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5 3: I1113 11:04:56.084278 17322 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5' 3: I1113 11:04:56.084460 17340 authenticator.cpp:204] Received SASL authentication start 3: I1113 11:04:56.084520 17340 authenticator.cpp:326] Authentication requires more steps 3: I1113 11:04:56.084689 17337 authenticatee.cpp:259] Received SASL authentication step 3: I1113 11:04:56.084872 17331 authenticator.cpp:232] Received SASL authentication step 3: I1113 11:04:56.084913 17331 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: '08347a511b13' server FQDN: '08347a511b13' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false 3: I1113 11:04:56.084929 17331 auxprop.cpp:181] Looking up auxiliary property '*userPassword' 3: I1113 11:04:56.084959 17331 auxprop.cpp:181] Looking up auxiliary property '*cmusaslsecretCRAM-MD5' 3: I1113 11:04:56.084978 17331 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: '08347a511b13' server FQDN: '08347a511b13' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true 3: I1113 11:04:56.084995 17331 auxprop.cpp:131] Skipping auxiliary property '*userPassword' since SASL_AUXPROP_AUTHZID == true 3: I1113 11:04:56.085003 17331 auxprop.cpp:131] Skipping auxiliary property '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true 3: I1113 11:04:56.085018 17331 authenticator.cpp:318] Authentication success 3: I1113 11:04:56.085150 17332 authenticatee.cpp:299] Authentication success 3: I1113 11:04:56.085181 17324 master.cpp:8316] Successfully authenticated principal 'test-principal' at [email protected]:44093 3: I1113 11:04:56.085227 17343 authenticator.cpp:432] Authentication session cleanup for crammd5-authenticatee(1088)@172.17.0.2:44093 3: I1113 11:04:56.085448 17332 sched.cpp:502] Successfully authenticated with master [email protected]:44093 3: I1113 11:04:56.085471 17332 sched.cpp:824] Sending SUBSCRIBE call to [email protected]:44093 3: I1113 11:04:56.085593 17332 sched.cpp:857] Will retry registration in 514.418393ms if necessary 3: I1113 11:04:56.085597 17323 hierarchical.cpp:2156] Filtered offer with cpus:1; mem:256; disk:367463; ports:[31000-32000]; mem(reservations: [(DYNAMIC,role1,test-principal)]):256 on agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 for role role1 of framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 3: I1113 11:04:56.085798 17323 hierarchical.cpp:1450] Performed allocation for 1 agents in 971460ns 3: I1113 11:04:56.085850 17338 master.cpp:2962] Received SUBSCRIBE call for framework 'framework2' at [email protected]:44093 3: I1113 11:04:56.085947 17338 master.cpp:2278] Authorizing framework principal 'test-principal' to receive offers for roles '{ role2 }' 3: I1113 11:04:56.086546 17334 master.cpp:3042] Subscribing framework framework2 with checkpointing disabled and capabilities [ RESERVATION_REFINEMENT ] 3: I1113 11:04:56.087180 17334 sched.cpp:751] Framework registered with 5f212a45-c32c-48f2-8392-e37909bcc50b-0001 3: I1113 11:04:56.087275 17334 sched.cpp:765] Scheduler::registered took 60654ns 3: I1113 11:04:56.087361 17341 hierarchical.cpp:303] Added framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0001 3: I1113 11:04:56.089180 17341 hierarchical.cpp:2156] Filtered offer with mem(reservations: [(DYNAMIC,role1,test-principal)]):256 on agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 for role role1 of framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 3: I1113 11:04:56.089457 17341 hierarchical.cpp:1450] Performed allocation for 1 agents in 1.927961ms 3: I1113 11:04:56.090132 17327 master.cpp:8116] Sending 1 offers to framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0001 (framework2) at [email protected]:44093 3: I1113 11:04:56.090911 17329 sched.cpp:921] Scheduler::resourceOffers took 163566ns 3: I1113 11:04:56.092013 17340 hierarchical.cpp:2156] Filtered offer with mem(reservations: [(DYNAMIC,role1,test-principal)]):256 on agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 for role role1 of framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 3: I1113 11:04:56.092391 17340 hierarchical.cpp:1450] Performed allocation for 1 agents in 899041ns 3: I1113 11:04:56.093272 17337 master.cpp:10016] Removing offer 5f212a45-c32c-48f2-8392-e37909bcc50b-O2 3: I1113 11:04:56.093452 17337 master.cpp:4229] Processing ACCEPT call for offers: [ 5f212a45-c32c-48f2-8392-e37909bcc50b-O2 ] on agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 at slave(540)@172.17.0.2:44093 (08347a511b13) for framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0001 (framework2) at [email protected]:44093 3: I1113 11:04:56.093672 17337 master.cpp:3724] Authorizing principal 'test-principal' to unreserve resources '[{"allocation_info":{"role":"role2"},"name":"mem","reservations":[{"principal":"test-principal","role":"role1","type":"DYNAMIC"}],"scalar":{"value":256.0},"type":"SCALAR"}]' 3: W1113 11:04:56.095199 17337 master.cpp:2363] Dropping UNRESERVE offer operation from framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0001 (framework2) at [email protected]:44093: cpus(allocated: role2):1; mem(allocated: role2):256; disk(allocated: role2):367463; ports(allocated: role2):[31000-32000] does not contain mem(allocated: role2)(reservations: [(DYNAMIC,role1,test-principal)]):256 3: I1113 11:04:56.096915 17332 hierarchical.cpp:1125] Recovered cpus(allocated: role2):1; mem(allocated: role2):256; disk(allocated: role2):367463; ports(allocated: role2):[31000-32000] (total: cpus:1; mem:256; disk:367463; ports:[31000-32000]; mem(reservations: [(DYNAMIC,role1,test-principal)]):256, allocated: {}) on agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 from framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0001 3: I1113 11:04:56.099956 17323 hierarchical.cpp:2156] Filtered offer with mem(reservations: [(DYNAMIC,role1,test-principal)]):256 on agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 for role role1 of framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 3: I1113 11:04:56.100239 17323 hierarchical.cpp:1450] Performed allocation for 1 agents in 2.041144ms 3: I1113 11:04:56.100904 17338 master.cpp:8116] Sending 1 offers to framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0001 (framework2) at [email protected]:44093 3: I1113 11:04:56.101584 17335 sched.cpp:921] Scheduler::resourceOffers took 152073ns 3: I1113 11:04:56.102212 17320 sched.cpp:2009] Asked to stop the driver 3: W1113 11:04:56.102294 17327 sched.cpp:1402] Attempting to decline an unknown offer 5f212a45-c32c-48f2-8392-e37909bcc50b-O2 3: I1113 11:04:56.102581 17327 sched.cpp:1191] Stopping framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0001 3: I1113 11:04:56.102706 17329 master.cpp:5433] Processing DECLINE call for offers: [ 5f212a45-c32c-48f2-8392-e37909bcc50b-O2 ] for framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0001 (framework2) at [email protected]:44093 3: W1113 11:04:56.102767 17329 master.cpp:5454] Ignoring decline of offer 5f212a45-c32c-48f2-8392-e37909bcc50b-O2 since it is no longer valid 3: I1113 11:04:56.102962 17329 master.cpp:8793] Processing TEARDOWN call for framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0001 (framework2) at [email protected]:44093 3: I1113 11:04:56.103003 17329 master.cpp:8805] Removing framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0001 (framework2) at [email protected]:44093 3: I1113 11:04:56.103029 17329 master.cpp:3332] Deactivating framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0001 (framework2) at [email protected]:44093 3: I1113 11:04:56.103229 17336 hierarchical.cpp:412] Deactivated framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0001 3: I1113 11:04:56.103942 17329 master.cpp:10016] Removing offer 5f212a45-c32c-48f2-8392-e37909bcc50b-O3 3: I1113 11:04:56.104189 17340 slave.cpp:3267] Asked to shut down framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0001 by [email protected]:44093 3: I1113 11:04:56.104233 17340 slave.cpp:3282] Cannot shut down unknown framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0001 3: I1113 11:04:56.104579 17329 master.cpp:5538] Processing REVIVE call for framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 (framework1) at [email protected]:44093 3: I1113 11:04:56.104774 17322 hierarchical.cpp:1125] Recovered cpus(allocated: role2):1; mem(allocated: role2):256; disk(allocated: role2):367463; ports(allocated: role2):[31000-32000] (total: cpus:1; mem:256; disk:367463; ports:[31000-32000]; mem(reservations: [(DYNAMIC,role1,test-principal)]):256, allocated: {}) on agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 from framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0001 3: I1113 11:04:56.105232 17322 hierarchical.cpp:355] Removed framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0001 3: I1113 11:04:56.105370 17322 hierarchical.cpp:1272] Revived offers for roles { role1 } of framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 3: I1113 11:04:56.107462 17322 hierarchical.cpp:1450] Performed allocation for 1 agents in 1.906565ms 3: I1113 11:04:56.107929 17322 hierarchical.cpp:1450] Performed allocation for 1 agents in 210289ns 3: I1113 11:04:56.108428 17325 master.cpp:8116] Sending 1 offers to framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 (framework1) at [email protected]:44093 3: I1113 11:04:56.109266 17321 sched.cpp:921] Scheduler::resourceOffers took 182029ns 3: I1113 11:04:56.109973 17320 sched.cpp:2009] Asked to stop the driver 3: I1113 11:04:56.110131 17323 sched.cpp:1191] Stopping framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 3: I1113 11:04:56.110447 17338 master.cpp:8793] Processing TEARDOWN call for framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 (framework1) at [email protected]:44093 3: I1113 11:04:56.110486 17338 master.cpp:8805] Removing framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 (framework1) at [email protected]:44093 3: I1113 11:04:56.110502 17338 master.cpp:3332] Deactivating framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 (framework1) at [email protected]:44093 3: I1113 11:04:56.110664 17344 hierarchical.cpp:412] Deactivated framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 3: I1113 11:04:56.111559 17338 master.cpp:10016] Removing offer 5f212a45-c32c-48f2-8392-e37909bcc50b-O4 3: I1113 11:04:56.111805 17322 slave.cpp:3267] Asked to shut down framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 by [email protected]:44093 3: I1113 11:04:56.111843 17322 slave.cpp:3282] Cannot shut down unknown framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 3: I1113 11:04:56.112618 17320 slave.cpp:883] Agent terminating 3: I1113 11:04:56.112821 17341 hierarchical.cpp:1125] Recovered cpus(allocated: role1):1; mem(allocated: role1):256; disk(allocated: role1):367463; ports(allocated: role1):[31000-32000]; mem(allocated: role1)(reservations: [(DYNAMIC,role1,test-principal)]):256 (total: cpus:1; mem:256; disk:367463; ports:[31000-32000]; mem(reservations: [(DYNAMIC,role1,test-principal)]):256, allocated: {}) on agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 from framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 3: I1113 11:04:56.112895 17324 master.cpp:1309] Agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 at slave(540)@172.17.0.2:44093 (08347a511b13) disconnected 3: I1113 11:04:56.112927 17324 master.cpp:3369] Disconnecting agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 at slave(540)@172.17.0.2:44093 (08347a511b13) 3: I1113 11:04:56.112994 17324 master.cpp:3388] Deactivating agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 at slave(540)@172.17.0.2:44093 (08347a511b13) 3: I1113 11:04:56.113332 17341 hierarchical.cpp:355] Removed framework 5f212a45-c32c-48f2-8392-e37909bcc50b-0000 3: I1113 11:04:56.113432 17341 hierarchical.cpp:690] Agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 deactivated 3: I1113 11:04:56.113833 17341 hierarchical.cpp:1450] Performed allocation for 1 agents in 171013ns 3: I1113 11:04:56.119732 17331 hierarchical.cpp:1450] Performed allocation for 1 agents in 186902ns 3: I1113 11:04:56.121114 17320 master.cpp:1151] Master terminating 3: I1113 11:04:56.122174 17344 hierarchical.cpp:626] Removed agent 5f212a45-c32c-48f2-8392-e37909bcc50b-S0 3: [ OK ] ReservationTest.PreventUnreservingAlienResources (117 ms) 3: [----------] 19 tests from ReservationTest (1870 ms total) 3: 3: [----------] 3 tests from SlaveStateTest 3: [ RUN ] SlaveStateTest.CheckpointString 3: [ OK ] SlaveStateTest.CheckpointString (0 ms) 3: [ RUN ] SlaveStateTest.CheckpointProtobufMessage 3: [ OK ] SlaveStateTest.CheckpointProtobufMessage (0 ms) 3: [ RUN ] SlaveStateTest.CheckpointRepeatedProtobufMessages 3: [ OK ] SlaveStateTest.CheckpointRepeatedProtobufMessages (2 ms) 3: [----------] 3 tests from SlaveStateTest (3 ms total) 3: 3: [----------] 29 tests from SlaveRecoveryTest/0, where TypeParam = mesos::internal::slave::MesosContainerizer 3: [ RUN ] SlaveRecoveryTest/0.RecoverSlaveState 3: I1113 11:04:56.134543 17320 cluster.cpp:162] Creating default 'local' authorizer 3: I1113 11:04:56.138557 17329 master.cpp:448] Master b050391c-f0f2-47be-a0b7-3e6b75eaf8ed (08347a511b13) started on 172.17.0.2:44093 3: I1113 11:04:56.138593 17329 master.cpp:450] Flags at startup: --acls="" --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins" --allocation_interval="1secs" --allocator="HierarchicalDRF" --authenticate_agents="true" --authenticate_frameworks="true" --authenticate_http_frameworks="true" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticators="crammd5" --authorizers="local" --credentials="/tmp/i0w58D/credentials" --filter_gpu_resources="true" --framework_sorter="drf" --help="false" --hostname_lookup="true" --http_authenticators="basic" --http_framework_authenticators="basic" --initialize_driver_logging="true" --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO" --max_agent_ping_timeouts="5" --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000" --max_unreachable_tasks_per_framework="1000" --port="5050" --quiet="false" --recovery_agent_removal_limit="100%" --registry="in_memory" --registry_fetch_timeout="1mins" --registry_gc_interval="15mins" --registry_max_agent_age="2weeks" --registry_max_agent_count="102400" --registry_store_timeout="100secs" --registry_strict="false" --root_submissions="true" --user_sorter="drf" --version="false" --webui_dir="/usr/local/share/mesos/webui" --work_dir="/tmp/i0w58D/master" --zk_session_timeout="10secs" 3: I1113 11:04:56.139060 17329 master.cpp:499] Master only allowing authenticated frameworks to register 3: I1113 11:04:56.139073 17329 master.cpp:505] Master only allowing authenticated agents to register 3: I1113 11:04:56.139081 17329 master.cpp:511] Master only allowing authenticated HTTP frameworks to register 3: I1113 11:04:56.139089 17329 credentials.hpp:37] Loading credentials for authentication from '/tmp/i0w58D/credentials' 3: I1113 11:04:56.139528 17329 master.cpp:555] Using default 'crammd5' authenticator 3: I1113 11:04:56.139749 17329 http.cpp:1045] Creating default 'basic' HTTP authenticator for realm 'mesos-master-readonly' 3: I1113 11:04:56.140007 17329 http.cpp:1045] Creating default 'basic' HTTP authenticator for realm 'mesos-master-readwrite' 3: I1113 11:04:56.140238 17329 http.cpp:1045] Creating default 'basic' HTTP authenticator for realm 'mesos-master-scheduler' 3: I1113 11:04:56.140502 17329 master.cpp:634] Authorization enabled 3: I1113 11:04:56.140763 17338 whitelist_watcher.cpp:77] No whitelist given 3: I1113 11:04:56.140781 17322 hierarchical.cpp:171] Initialized hierarchical allocator process 3: I1113 11:04:56.144371 17325 master.cpp:2213] Elected as the leading master! 3: I1113 11:04:56.144405 17325 master.cpp:1693] Recovering from registrar 3: I1113 11:04:56.144598 17328 registrar.cpp:347] Recovering registrar 3: I1113 11:04:56.145287 17328 registrar.cpp:391] Successfully fetched the registry (0B) in 641280ns 3: I1113 11:04:56.145407 17328 registrar.cpp:495] Applied 1 operations in 31272ns; attempting to update the registry 3: I1113 11:04:56.146064 17328 registrar.cpp:552] Successfully updated the registry in 595968ns 3: I1113 11:04:56.146205 17328 registrar.cpp:424] Successfully recovered registrar 3: I1113 11:04:56.146662 17321 master.cpp:1806] Recovered 0 agents from the registry (129B); allowing 10mins for agents to re-register 3: I1113 11:04:56.146759 17337 hierarchical.cpp:209] Skipping recovery of hierarchical allocator: nothing to recover 3: I1113 11:04:56.151181 17320 containerizer.cpp:301] Using isolation { environment_secret, posix/cpu, posix/mem, filesystem/posix, network/cni } 3: W1113 11:04:56.151914 17320 backend.cpp:76] Failed to create 'aufs' backend: AufsBackend requires root privileges 3: W1113 11:04:56.152091 17320 backend.cpp:76] Failed to create 'bind' backend: BindBackend requires root privileges 3: I1113 11:04:56.152140 17320 provisioner.cpp:255] Using default backend 'copy' 3: W1113 11:04:56.157500 17320 process.cpp:3178] Attempted to spawn already running process [email protected]:44093 3: I1113 11:04:56.157891 17320 cluster.cpp:448] Creating default 'local' authorizer 3: I1113 11:04:56.160696 17337 slave.cpp:262] Mesos agent started on (541)@172.17.0.2:44093 3: I1113 11:04:56.160728 17337 slave.cpp:263] Flags at startup: --acls="" --appc_simple_discovery_uri_prefix="http://"; --appc_store_dir="/tmp/SlaveRecoveryTest_0_RecoverSlaveState_C3PCZF/store/appc" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs" --containerizers="mesos" --credential="/tmp/SlaveRecoveryTest_0_RecoverSlaveState_C3PCZF/credential" --default_role="*" --disallow_sharing_agent_pid_namespace="false" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io"; --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/tmp/SlaveRecoveryTest_0_RecoverSlaveState_C3PCZF/store/docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_reregistration_timeout="2secs" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/SlaveRecoveryTest_0_RecoverSlaveState_C3PCZF/fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname_lookup="true" --http_command_executor="false" --http_credentials="/tmp/SlaveRecoveryTest_0_RecoverSlaveState_C3PCZF/http_credentials" --http_heartbeat_interval="30secs" --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" --launcher="posix" --launcher_dir="/mesos/build/src" --logbufsecs="0" --logging_level="INFO" --max_completed_executors_per_framework="150" --oversubscribed_resources_interval="15secs" --perf_duration="10secs" --perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true" --runtime_dir="/tmp/SlaveRecoveryTest_0_RecoverSlaveState_C3PCZF" --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" --systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/system" --version="false" --work_dir="/tmp/SlaveRecoveryTest_0_RecoverSlaveState_xuul49" --zk_session_timeout="10secs" 3: I1113 11:04:56.161411 17337 credentials.hpp:86] Loading credential for authentication from '/tmp/SlaveRecoveryTest_0_RecoverSlaveState_C3PCZF/credential' 3: W1113 11:04:56.161447 17320 process.cpp:3178] Attempted to spawn already running process [email protected]:44093 3: I1113 11:04:56.161655 17337 slave.cpp:295] Agent using credential for: test-principal 3: I1113 11:04:56.161686 17337 credentials.hpp:37] Loading credentials for authentication from '/tmp/SlaveRecoveryTest_0_RecoverSlaveState_C3PCZF/http_credentials' 3: I1113 11:04:56.162061 17337 http.cpp:1045] Creating default 'basic' HTTP authenticator for realm 'mesos-agent-readonly' 3: I1113 11:04:56.162297 17337 http.cpp:1045] Creating default 'basic' HTTP authenticator for realm 'mesos-agent-readwrite' 3: I1113 11:04:56.162693 17320 sched.cpp:232] Version: 1.5.0 3: I1113 11:04:56.163316 17333 sched.cpp:336] New master detected at [email protected]:44093 3: I1113 11:04:56.163447 17333 sched.cpp:396] Authenticating with master [email protected]:44093 3: I1113 11:04:56.163470 17333 sched.cpp:403] Using default CRAM-MD5 authenticatee 3: I1113 11:04:56.163797 17324 authenticatee.cpp:121] Creating new client SASL connection 3: I1113 11:04:56.164183 17332 master.cpp:8286] Authenticating [email protected]:44093 3: I1113 11:04:56.164407 17331 authenticator.cpp:414] Starting authentication session for crammd5-authenticatee(1089)@172.17.0.2:44093 3: I1113 11:04:56.164746 17328 authenticator.cpp:98] Creating new server SASL connection 3: I1113 11:04:56.164573 17337 slave.cpp:593] Agent resources: [{"name":"cpus","scalar":{"value":2.0},"type":"SCALAR"},{"name":"mem","scalar":{"value":1024.0},"type":"SCALAR"},{"name":"disk","scalar":{"value":1024.0},"type":"SCALAR"},{"name":"ports","ranges":{"range":[{"begin":31000,"end":32000}]},"type":"RANGES"}] 3: I1113 11:04:56.164947 17337 slave.cpp:601] Agent attributes: [ ] 3: I1113 11:04:56.164961 17337 slave.cpp:610] Agent hostname: 08347a511b13 3: I1113 11:04:56.165086 17335 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5 3: I1113 11:04:56.165129 17335 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5' 3: I1113 11:04:56.165171 17334 status_update_manager.cpp:177] Pausing sending status updates 3: I1113 11:04:56.165305 17323 authenticator.cpp:204] Received SASL authentication start 3: I1113 11:04:56.165392 17323 authenticator.cpp:326] Authentication requires more steps 3: I1113 11:04:56.165549 17323 authenticatee.cpp:259] Received SASL authentication step 3: I1113 11:04:56.165751 17338 authenticator.cpp:232] Received SASL authentication step 3: I1113 11:04:56.165798 17338 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: '08347a511b13' server FQDN: '08347a511b13' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false 3: I1113 11:04:56.165813 17338 auxprop.cpp:181] Looking up auxiliary property '*userPassword' 3: I1113 11:04:56.165853 17338 auxprop.cpp:181] Looking up auxiliary property '*cmusaslsecretCRAM-MD5' 3: I1113 11:04:56.165885 17338 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: '08347a511b13' server FQDN: '08347a511b13' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true 3: I1113 11:04:56.165906 17338 auxprop.cpp:131] Skipping auxiliary property '*userPassword' since SASL_AUXPROP_AUTHZID == true 3: I1113 11:04:56.165915 17338 auxprop.cpp:131] Skipping auxiliary property '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true 3: I1113 11:04:56.165933 17338 authenticator.cpp:318] Authentication success 3: I1113 11:04:56.166096 17329 authenticatee.cpp:299] Authentication success 3: I1113 11:04:56.166210 17325 authenticator.cpp:432] Authentication session cleanup for crammd5-authenticatee(1089)@172.17.0.2:44093 3: I1113 11:04:56.166272 17339 master.cpp:8316] Successfully authenticated principal 'test-principal' at [email protected]:44093 3: I1113 11:04:56.166532 17329 sched.cpp:502] Successfully authenticated with master [email protected]:44093 3: I1113 11:04:56.166565 17329 sched.cpp:824] Sending SUBSCRIBE call to [email protected]:44093 3: I1113 11:04:56.166738 17329 sched.cpp:857] Will retry registration in 713.326473ms if necessary 3: I1113 11:04:56.167109 17327 master.cpp:2962] Received SUBSCRIBE call for framework 'default' at [email protected]:44093 3: I1113 11:04:56.167228 17327 master.cpp:2278] Authorizing framework principal 'test-principal' to receive offers for roles '{ * }' 3: I1113 11:04:56.167359 17341 state.cpp:64] Recovering state from '/tmp/SlaveRecoveryTest_0_RecoverSlaveState_xuul49/meta' 3: I1113 11:04:56.167721 17328 status_update_manager.cpp:203] Recovering status update manager 3: I1113 11:04:56.167876 17335 master.cpp:3042] Subscribing framework default with checkpointing enabled and capabilities [ RESERVATION_REFINEMENT ] 3: I1113 11:04:56.168004 17321 containerizer.cpp:662] Recovering containerizer 3: I1113 11:04:56.168779 17344 sched.cpp:751] Framework registered with b050391c-f0f2-47be-a0b7-3e6b75eaf8ed-0000 write /dev/stdout: resource temporarily unavailable + docker rmi mesos-1510568564-294 Build step 'Execute shell' marked build as failure
