See <https://builds.apache.org/job/Mesos-Buildbot/BUILDTOOL=cmake,COMPILER=clang,CONFIGURATION=--verbose%20--disable-libtool-wrappers,ENVIRONMENT=GLOG_v=1%20MESOS_VERBOSE=1,OS=ubuntu%3A14.04,label_exp=(ubuntu)&&(!ubuntu-us1)&&(!ubuntu-eu2)&&(!qnode3)&&(!H23)/4578/display/redirect?page=changes>
Changes: [yujie.jay] Refactored agent to keep track of local resource providers. [yujie.jay] Passed the message directly to the Master::registerSlave handler. [yujie.jay] Sent resource version uuid only for agent default resources. ------------------------------------------ [...truncated 5.18 MB...] 3: I1211 23:37:31.522328 16001 hierarchical.cpp:1513] Performed allocation for 0 agents in 88725ns 3: I1211 23:37:31.523797 16005 provisioner.cpp:495] Provisioner recovery complete 3: I1211 23:37:31.524212 16006 slave.cpp:6727] Finished recovery 3: I1211 23:37:31.525128 16002 task_status_update_manager.cpp:181] Pausing sending task status updates 3: I1211 23:37:31.525147 16007 slave.cpp:1033] New master detected at [email protected]:50774 3: I1211 23:37:31.525265 16007 slave.cpp:1088] Detecting new master 3: I1211 23:37:31.535737 16010 slave.cpp:1115] Authenticating with master [email protected]:50774 3: I1211 23:37:31.535804 16010 slave.cpp:1124] Using default CRAM-MD5 authenticatee 3: I1211 23:37:31.536082 15998 authenticatee.cpp:121] Creating new client SASL connection 3: I1211 23:37:31.536396 15999 master.cpp:8888] Authenticating slave(100)@172.17.0.5:50774 3: I1211 23:37:31.536515 16004 authenticator.cpp:414] Starting authentication session for crammd5-authenticatee(208)@172.17.0.5:50774 3: I1211 23:37:31.536772 16009 authenticator.cpp:98] Creating new server SASL connection 3: I1211 23:37:31.536999 16011 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5 3: I1211 23:37:31.537034 16011 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5' 3: I1211 23:37:31.537163 15997 authenticator.cpp:204] Received SASL authentication start 3: I1211 23:37:31.537212 15997 authenticator.cpp:326] Authentication requires more steps 3: I1211 23:37:31.537331 16003 authenticatee.cpp:259] Received SASL authentication step 3: I1211 23:37:31.537472 16005 authenticator.cpp:232] Received SASL authentication step 3: I1211 23:37:31.537508 16005 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: 'da72a4ac167e' server FQDN: 'da72a4ac167e' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false 3: I1211 23:37:31.537523 16005 auxprop.cpp:181] Looking up auxiliary property '*userPassword' 3: I1211 23:37:31.537561 16005 auxprop.cpp:181] Looking up auxiliary property '*cmusaslsecretCRAM-MD5' 3: I1211 23:37:31.537590 16005 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: 'da72a4ac167e' server FQDN: 'da72a4ac167e' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true 3: I1211 23:37:31.537603 16005 auxprop.cpp:131] Skipping auxiliary property '*userPassword' since SASL_AUXPROP_AUTHZID == true 3: I1211 23:37:31.537611 16005 auxprop.cpp:131] Skipping auxiliary property '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true 3: I1211 23:37:31.537626 16005 authenticator.cpp:318] Authentication success 3: I1211 23:37:31.537714 16008 authenticatee.cpp:299] Authentication success 3: I1211 23:37:31.537844 16006 master.cpp:8918] Successfully authenticated principal 'test-principal' at slave(100)@172.17.0.5:50774 3: I1211 23:37:31.537899 16001 authenticator.cpp:432] Authentication session cleanup for crammd5-authenticatee(208)@172.17.0.5:50774 3: I1211 23:37:31.538126 15996 slave.cpp:1207] Successfully authenticated with master [email protected]:50774 3: I1211 23:37:31.538399 15996 slave.cpp:1684] Will retry registration in 16.499348ms if necessary 3: I1211 23:37:31.538607 15998 master.cpp:6077] Received register agent message from slave(100)@172.17.0.5:50774 (da72a4ac167e) 3: I1211 23:37:31.538656 15998 master.cpp:3866] Authorizing agent with principal 'test-principal' 3: I1211 23:37:31.539074 15999 master.cpp:6147] Authorized registration of agent at slave(100)@172.17.0.5:50774 (da72a4ac167e) 3: I1211 23:37:31.539175 15999 master.cpp:6240] Registering agent at slave(100)@172.17.0.5:50774 (da72a4ac167e) with id 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 3: I1211 23:37:31.539664 16009 registrar.cpp:495] Applied 1 operations in 107787ns; attempting to update the registry 3: I1211 23:37:31.540294 16009 registrar.cpp:552] Successfully updated the registry in 571904ns 3: I1211 23:37:31.540455 16005 master.cpp:6288] Admitted agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 at slave(100)@172.17.0.5:50774 (da72a4ac167e) 3: I1211 23:37:31.541102 16005 master.cpp:6337] Registered agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 at slave(100)@172.17.0.5:50774 (da72a4ac167e) with cpus:2; mem:1024; disk:1024; ports:[31000-32000] 3: I1211 23:37:31.541240 16007 slave.cpp:1253] Registered with master [email protected]:50774; given agent ID 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 3: I1211 23:37:31.541370 16010 task_status_update_manager.cpp:188] Resuming sending task status updates 3: I1211 23:37:31.541648 16007 slave.cpp:1273] Checkpointing SlaveInfo to '/tmp/MasterSlaveReconciliationTest_ReconcileLostTask_Y4MbkZ/meta/slaves/8c67f63c-1984-4337-81a4-32ebec5557ac-S0/slave.info' 3: I1211 23:37:31.541720 16000 hierarchical.cpp:572] Added agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 (da72a4ac167e) with cpus:2; mem:1024; disk:1024; ports:[31000-32000] (allocated: {}) 3: I1211 23:37:31.542165 16007 slave.cpp:1330] Forwarding agent update {"offer_operations":{},"resource_categories":{"oversubscribed":true},"resource_version_uuid":"hl9cTU+rR\/S0Ed1PsVi\/Qg==","slave_id":{"value":"8c67f63c-1984-4337-81a4-32ebec5557ac-S0"}} 3: I1211 23:37:31.542490 15996 master.cpp:7238] Received update of agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 at slave(100)@172.17.0.5:50774 (da72a4ac167e) with total oversubscribed resources {} 3: I1211 23:37:31.542783 15996 master.cpp:7336] Ignoring update on agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 at slave(100)@172.17.0.5:50774 (da72a4ac167e) as it reports no changes 3: I1211 23:37:31.543519 16000 hierarchical.cpp:1513] Performed allocation for 1 agents in 1.605756ms 3: I1211 23:37:31.544028 15998 master.cpp:8718] Sending 1 offers to framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 (default) at [email protected]:50774 3: I1211 23:37:31.544589 16004 sched.cpp:921] Scheduler::resourceOffers took 131419ns 3: I1211 23:37:31.546694 16002 master.cpp:10638] Removing offer 8c67f63c-1984-4337-81a4-32ebec5557ac-O0 3: I1211 23:37:31.546878 16002 master.cpp:4273] Processing ACCEPT call for offers: [ 8c67f63c-1984-4337-81a4-32ebec5557ac-O0 ] on agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 at slave(100)@172.17.0.5:50774 (da72a4ac167e) for framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 (default) at [email protected]:50774 3: I1211 23:37:31.547024 16002 master.cpp:3593] Authorizing framework principal 'test-principal' to launch task 1 3: W1211 23:37:31.549165 16001 validation.cpp:1387] Executor 'default' for task '1' uses less CPUs (None) than the minimum required (0.01). Please update your executor, as this will be mandatory in future releases. 3: W1211 23:37:31.549199 16001 validation.cpp:1399] Executor 'default' for task '1' uses less memory (None) than the minimum required (32MB). Please update your executor, as this will be mandatory in future releases. 3: I1211 23:37:31.549862 16001 master.cpp:11393] Adding task 1 with resources cpus(allocated: *):2; mem(allocated: *):1024; disk(allocated: *):1024; ports(allocated: *):[31000-32000] on agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 at slave(100)@172.17.0.5:50774 (da72a4ac167e) 3: I1211 23:37:31.550323 16001 master.cpp:5024] Launching task 1 of framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 (default) at [email protected]:50774 with resources [{"allocation_info":{"role":"*"},"name":"cpus","scalar":{"value":2.0},"type":"SCALAR"},{"allocation_info":{"role":"*"},"name":"mem","scalar":{"value":1024.0},"type":"SCALAR"},{"allocation_info":{"role":"*"},"name":"disk","scalar":{"value":1024.0},"type":"SCALAR"},{"allocation_info":{"role":"*"},"name":"ports","ranges":{"range":[{"begin":31000,"end":32000}]},"type":"RANGES"}] on agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 at slave(100)@172.17.0.5:50774 (da72a4ac167e) 3: I1211 23:37:31.552305 16009 slave.cpp:1033] New master detected at [email protected]:50774 3: I1211 23:37:31.552333 16006 task_status_update_manager.cpp:181] Pausing sending task status updates 3: I1211 23:37:31.552366 16009 slave.cpp:1088] Detecting new master 3: I1211 23:37:31.561091 16005 slave.cpp:1115] Authenticating with master [email protected]:50774 3: I1211 23:37:31.561151 16005 slave.cpp:1124] Using default CRAM-MD5 authenticatee 3: I1211 23:37:31.561378 16002 authenticatee.cpp:121] Creating new client SASL connection 3: I1211 23:37:31.561703 16010 master.cpp:8888] Authenticating slave(100)@172.17.0.5:50774 3: I1211 23:37:31.561873 16007 authenticator.cpp:414] Starting authentication session for crammd5-authenticatee(209)@172.17.0.5:50774 3: I1211 23:37:31.562173 16001 authenticator.cpp:98] Creating new server SASL connection 3: I1211 23:37:31.562451 15996 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5 3: I1211 23:37:31.562479 15996 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5' 3: I1211 23:37:31.562608 15996 authenticator.cpp:204] Received SASL authentication start 3: I1211 23:37:31.562671 15996 authenticator.cpp:326] Authentication requires more steps 3: I1211 23:37:31.562805 16000 authenticatee.cpp:259] Received SASL authentication step 3: I1211 23:37:31.562950 16004 authenticator.cpp:232] Received SASL authentication step 3: I1211 23:37:31.562988 16004 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: 'da72a4ac167e' server FQDN: 'da72a4ac167e' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false 3: I1211 23:37:31.562999 16004 auxprop.cpp:181] Looking up auxiliary property '*userPassword' 3: I1211 23:37:31.563030 16004 auxprop.cpp:181] Looking up auxiliary property '*cmusaslsecretCRAM-MD5' 3: I1211 23:37:31.563069 16004 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: 'da72a4ac167e' server FQDN: 'da72a4ac167e' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true 3: I1211 23:37:31.563082 16004 auxprop.cpp:131] Skipping auxiliary property '*userPassword' since SASL_AUXPROP_AUTHZID == true 3: I1211 23:37:31.563089 16004 auxprop.cpp:131] Skipping auxiliary property '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true 3: I1211 23:37:31.563104 16004 authenticator.cpp:318] Authentication success 3: I1211 23:37:31.563199 15998 authenticatee.cpp:299] Authentication success 3: I1211 23:37:31.563314 16003 master.cpp:8918] Successfully authenticated principal 'test-principal' at slave(100)@172.17.0.5:50774 3: I1211 23:37:31.563354 15999 authenticator.cpp:432] Authentication session cleanup for crammd5-authenticatee(209)@172.17.0.5:50774 3: I1211 23:37:31.563551 15998 slave.cpp:1207] Successfully authenticated with master [email protected]:50774 3: I1211 23:37:31.563778 15998 slave.cpp:1684] Will retry registration in 15.586254ms if necessary 3: I1211 23:37:31.563998 16002 master.cpp:6416] Received re-register agent message from agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 at slave(100)@172.17.0.5:50774 (da72a4ac167e) 3: I1211 23:37:31.564047 16002 master.cpp:3866] Authorizing agent with principal 'test-principal' 3: I1211 23:37:31.564429 16007 master.cpp:6505] Authorized re-registration of agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 at slave(100)@172.17.0.5:50774 (da72a4ac167e) 3: I1211 23:37:31.564529 16007 master.cpp:6571] Agent is already marked as registered: 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 at slave(100)@172.17.0.5:50774 (da72a4ac167e) 3: I1211 23:37:31.564826 16007 master.cpp:6916] Registry updated for slave 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 at slave(100)@172.17.0.5:50774(da72a4ac167e) 3: W1211 23:37:31.565230 16007 master.cpp:8981] Task 1 of framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 unknown to the agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 at slave(100)@172.17.0.5:50774 (da72a4ac167e) during re-registration: reconciling with the agent 3: I1211 23:37:31.565480 16001 hierarchical.cpp:665] Agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 (da72a4ac167e) updated with total resources cpus:2; mem:1024; disk:1024; ports:[31000-32000] 3: W1211 23:37:31.565629 16007 master.cpp:9048] Executor 'default' of framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 possibly unknown to the agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 at slave(100)@172.17.0.5:50774 (da72a4ac167e) 3: I1211 23:37:31.565723 16007 master.cpp:10249] Removing executor 'default' with resources [] of framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 on agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 at slave(100)@172.17.0.5:50774 (da72a4ac167e) 3: I1211 23:37:31.565963 16007 master.cpp:7077] Sending updated checkpointed resources {} to agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 at slave(100)@172.17.0.5:50774 (da72a4ac167e) 3: I1211 23:37:31.566032 16011 slave.cpp:1366] Re-registered with master [email protected]:50774 3: I1211 23:37:31.566151 16008 task_status_update_manager.cpp:188] Resuming sending task status updates 3: I1211 23:37:31.566262 16011 slave.cpp:1411] Forwarding agent update {"offer_operations":{},"resource_categories":{"oversubscribed":true},"resource_version_uuid":"hl9cTU+rR\/S0Ed1PsVi\/Qg==","slave_id":{"value":"8c67f63c-1984-4337-81a4-32ebec5557ac-S0"}} 3: W1211 23:37:31.566452 16011 slave.cpp:1451] Agent reconciling task 1 of framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 in state TASK_LOST: task unknown to the agent 3: I1211 23:37:31.566593 15999 master.cpp:7238] Received update of agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 at slave(100)@172.17.0.5:50774 (da72a4ac167e) with total oversubscribed resources {} 3: I1211 23:37:31.566646 15998 task_status_update_manager.cpp:328] Received task status update TASK_LOST (Status UUID: 4e1cde39-0cc2-4574-b3f4-979385828050) for task 1 of framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 3: I1211 23:37:31.566699 15998 task_status_update_manager.cpp:507] Creating StatusUpdate stream for task 1 of framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 3: I1211 23:37:31.566792 16011 slave.cpp:3624] Ignoring new checkpointed resources identical to the current version: {} 3: I1211 23:37:31.566912 15999 master.cpp:7336] Ignoring update on agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 at slave(100)@172.17.0.5:50774 (da72a4ac167e) as it reports no changes 3: I1211 23:37:31.567189 15998 task_status_update_manager.cpp:383] Forwarding task status update TASK_LOST (Status UUID: 4e1cde39-0cc2-4574-b3f4-979385828050) for task 1 of framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 to the agent 3: I1211 23:37:31.567351 16002 slave.cpp:5215] Forwarding the update TASK_LOST (Status UUID: 4e1cde39-0cc2-4574-b3f4-979385828050) for task 1 of framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 to [email protected]:50774 3: I1211 23:37:31.567795 16002 slave.cpp:5108] Task status update manager successfully handled status update TASK_LOST (Status UUID: 4e1cde39-0cc2-4574-b3f4-979385828050) for task 1 of framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 3: I1211 23:37:31.567968 16010 master.cpp:7811] Status update TASK_LOST (Status UUID: 4e1cde39-0cc2-4574-b3f4-979385828050) for task 1 of framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 from agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 at slave(100)@172.17.0.5:50774 (da72a4ac167e) 3: I1211 23:37:31.568032 16010 master.cpp:7867] Forwarding status update TASK_LOST (Status UUID: 4e1cde39-0cc2-4574-b3f4-979385828050) for task 1 of framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 3: I1211 23:37:31.568240 16010 master.cpp:10114] Updating the state of task 1 of framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 (latest state: TASK_LOST, status update state: TASK_LOST) 3: I1211 23:37:31.568462 16006 sched.cpp:1029] Scheduler::statusUpdate took 85193ns 3: I1211 23:37:31.569242 16010 master.cpp:5842] Processing ACKNOWLEDGE call 4e1cde39-0cc2-4574-b3f4-979385828050 for task 1 of framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 (default) at [email protected]:50774 on agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 3: I1211 23:37:31.569403 16010 master.cpp:10220] Removing task 1 with resources cpus(allocated: *):2; mem(allocated: *):1024; disk(allocated: *):1024; ports(allocated: *):[31000-32000] of framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 on agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 at slave(100)@172.17.0.5:50774 (da72a4ac167e) 3: I1211 23:37:31.569816 16000 hierarchical.cpp:1188] Recovered cpus(allocated: *):2; mem(allocated: *):1024; disk(allocated: *):1024; ports(allocated: *):[31000-32000] (total: cpus:2; mem:1024; disk:1024; ports:[31000-32000], allocated: {}) on agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 from framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 3: I1211 23:37:31.569936 16000 task_status_update_manager.cpp:401] Received task status update acknowledgement (UUID: 4e1cde39-0cc2-4574-b3f4-979385828050) for task 1 of framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 3: I1211 23:37:31.570051 16000 task_status_update_manager.cpp:538] Cleaning up status update stream for task 1 of framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 3: I1211 23:37:31.570441 15996 slave.cpp:4010] Task status update manager successfully handled status update acknowledgement (UUID: 4e1cde39-0cc2-4574-b3f4-979385828050) for task 1 of framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 3: E1211 23:37:31.570469 15996 slave.cpp:4021] Status update acknowledgement (UUID: 4e1cde39-0cc2-4574-b3f4-979385828050) for task 1 of unknown framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 3: I1211 23:37:31.572738 15997 process.cpp:3503] Handling HTTP event for process 'metrics' with path: '/metrics/snapshot' 3: I1211 23:37:31.573933 16006 http.cpp:851] Authorizing principal 'ANY' to GET the endpoint '/metrics/snapshot' 3: I1211 23:37:31.594840 15995 sched.cpp:2009] Asked to stop the driver 3: I1211 23:37:31.595023 16002 sched.cpp:1191] Stopping framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 3: I1211 23:37:31.595316 15996 master.cpp:9398] Processing TEARDOWN call for framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 (default) at [email protected]:50774 3: I1211 23:37:31.595345 15996 master.cpp:9410] Removing framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 (default) at [email protected]:50774 3: I1211 23:37:31.595365 15996 master.cpp:3327] Deactivating framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 (default) at [email protected]:50774 3: I1211 23:37:31.595528 16010 hierarchical.cpp:405] Deactivated framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 3: I1211 23:37:31.595584 16001 slave.cpp:3375] Asked to shut down framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 by [email protected]:50774 3: I1211 23:37:31.595618 16001 slave.cpp:3390] Cannot shut down unknown framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 3: I1211 23:37:31.595963 16008 hierarchical.cpp:344] Removed framework 8c67f63c-1984-4337-81a4-32ebec5557ac-0000 3: I1211 23:37:31.596173 15995 slave.cpp:909] Agent terminating 3: I1211 23:37:31.596346 15998 master.cpp:1305] Agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 at slave(100)@172.17.0.5:50774 (da72a4ac167e) disconnected 3: I1211 23:37:31.596371 15998 master.cpp:3364] Disconnecting agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 at slave(100)@172.17.0.5:50774 (da72a4ac167e) 3: I1211 23:37:31.596421 15998 master.cpp:3383] Deactivating agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 at slave(100)@172.17.0.5:50774 (da72a4ac167e) 3: I1211 23:37:31.596634 16011 hierarchical.cpp:762] Agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 deactivated 3: I1211 23:37:31.603332 15995 master.cpp:1147] Master terminating 3: I1211 23:37:31.604122 16005 hierarchical.cpp:605] Removed agent 8c67f63c-1984-4337-81a4-32ebec5557ac-S0 3: [ OK ] MasterSlaveReconciliationTest.ReconcileLostTask (117 ms) 3: [ RUN ] MasterSlaveReconciliationTest.ReconcileDroppedTask 3: I1211 23:37:31.610566 15995 cluster.cpp:172] Creating default 'local' authorizer 3: I1211 23:37:31.612957 16010 master.cpp:456] Master 156d40d2-e105-4862-ba03-3fdd2fe950ce (da72a4ac167e) started on 172.17.0.5:50774 3: I1211 23:37:31.612987 16010 master.cpp:458] Flags at startup: --acls="" --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins" --allocation_interval="1secs" --allocator="HierarchicalDRF" --authenticate_agents="true" --authenticate_frameworks="true" --authenticate_http_frameworks="true" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticators="crammd5" --authorizers="local" --credentials="/tmp/SAq5Mo/credentials" --filter_gpu_resources="true" --framework_sorter="drf" --help="false" --hostname_lookup="true" --http_authenticators="basic" --http_framework_authenticators="basic" --initialize_driver_logging="true" --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO" --max_agent_ping_timeouts="5" --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000" --max_unreachable_tasks_per_framework="1000" --port="5050" --quiet="false" --recovery_agent_removal_limit="100%" --registry="in_memory" --registry_fetch_timeout="1mins" --registry_gc_interval="15mins" --registry_max_agent_age="2weeks" --registry_max_agent_count="102400" --registry_store_timeout="100secs" --registry_strict="false" --root_submissions="true" --user_sorter="drf" --version="false" --webui_dir="/usr/local/share/mesos/webui" --work_dir="/tmp/SAq5Mo/master" --zk_session_timeout="10secs" 3: I1211 23:37:31.613271 16010 master.cpp:507] Master only allowing authenticated frameworks to register 3: I1211 23:37:31.613281 16010 master.cpp:513] Master only allowing authenticated agents to register 3: I1211 23:37:31.613287 16010 master.cpp:519] Master only allowing authenticated HTTP frameworks to register 3: I1211 23:37:31.613296 16010 credentials.hpp:37] Loading credentials for authentication from '/tmp/SAq5Mo/credentials' 3: I1211 23:37:31.613538 16010 master.cpp:563] Using default 'crammd5' authenticator 3: I1211 23:37:31.613672 16010 http.cpp:1045] Creating default 'basic' HTTP authenticator for realm 'mesos-master-readonly' 3: I1211 23:37:31.613826 16010 http.cpp:1045] Creating default 'basic' HTTP authenticator for realm 'mesos-master-readwrite' 3: I1211 23:37:31.613947 16010 http.cpp:1045] Creating default 'basic' HTTP authenticator for realm 'mesos-master-scheduler' 3: I1211 23:37:31.614070 16010 master.cpp:642] Authorization enabled 3: I1211 23:37:31.614212 16011 hierarchical.cpp:175] Initialized hierarchical allocator process 3: I1211 23:37:31.614266 16008 whitelist_watcher.cpp:77] No whitelist given 3: I1211 23:37:31.616423 15996 master.cpp:2209] Elected as the leading master! 3: I1211 23:37:31.616439 15996 master.cpp:1689] Recovering from registrar 3: I1211 23:37:31.616554 16001 registrar.cpp:347] Recovering registrar 3: I1211 23:37:31.617094 16001 registrar.cpp:391] Successfully fetched the registry (0B) in 517120ns 3: I1211 23:37:31.617192 16001 registrar.cpp:495] Applied 1 operations in 35907ns; attempting to update the registry 3: I1211 23:37:31.617652 16001 registrar.cpp:552] Successfully updated the registry in 416us 3: I1211 23:37:31.617761 16001 registrar.cpp:424] Successfully recovered registrar 3: I1211 23:37:31.618093 16000 master.cpp:1802] Recovered 0 agents from the registry (135B); allowing 10mins for agents to re-register 3: I1211 23:37:31.618136 15997 hierarchical.cpp:213] Skipping recovery of hierarchical allocator: nothing to recover 3: W1211 23:37:31.621768 15995 process.cpp:2756] Attempted to spawn already running process [email protected]:50774 3: I1211 23:37:31.622640 15995 containerizer.cpp:304] Using isolation { environment_secret, posix/cpu, posix/mem, filesystem/posix, network/cni } 3: W1211 23:37:31.623119 15995 backend.cpp:76] Failed to create 'aufs' backend: AufsBackend requires root privileges 3: W1211 23:37:31.623237 15995 backend.cpp:76] Failed to create 'bind' backend: BindBackend requires root privileges 3: I1211 23:37:31.623270 15995 provisioner.cpp:299] Using default backend 'copy' 3: I1211 23:37:31.625072 15995 cluster.cpp:460] Creating default 'local' authorizer 3: I1211 23:37:31.627279 16011 slave.cpp:258] Mesos agent started on (101)@172.17.0.5:50774 3: W1211 23:37:31.627624 15995 process.cpp:2756] Attempted to spawn already running process [email protected]:50774 3: I1211 23:37:31.627313 16011 slave.cpp:259] Flags at startup: --acls="" --appc_simple_discovery_uri_prefix="http://"; --appc_store_dir="/tmp/MasterSlaveReconciliationTest_ReconcileDroppedTask_wU5QUB/store/appc" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs" --containerizers="mesos" --credential="/tmp/MasterSlaveReconciliationTest_ReconcileDroppedTask_wU5QUB/credential" --default_role="*" --disallow_sharing_agent_pid_namespace="false" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io"; --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/tmp/MasterSlaveReconciliationTest_ReconcileDroppedTask_wU5QUB/store/docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_reregistration_timeout="2secs" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/MasterSlaveReconciliationTest_ReconcileDroppedTask_wU5QUB/fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname_lookup="true" --http_command_executor="false" --http_credentials="/tmp/MasterSlaveReconciliationTest_ReconcileDroppedTask_wU5QUB/http_credentials" --http_heartbeat_interval="30secs" --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" --launcher="posix" --launcher_dir="/mesos/build/src" --logbufsecs="0" --logging_level="INFO" --max_completed_executors_per_framework="150" --oversubscribed_resources_interval="15secs" --perf_duration="10secs" --perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns" --quiet="false" --reconfiguration_policy="equal" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true" --runtime_dir="/tmp/MasterSlaveReconciliationTest_ReconcileDroppedTask_wU5QUB" --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" --systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/system" --version="false" --work_dir="/tmp/MasterSlaveReconciliationTest_ReconcileDroppedTask_dNGXl0" --zk_session_timeout="10secs" 3: I1211 23:37:31.627677 16011 credentials.hpp:86] Loading credential for authentication from '/tmp/MasterSlaveReconciliationTest_ReconcileDroppedTask_wU5QUB/credential' 3: I1211 23:37:31.627830 16011 slave.cpp:291] Agent using credential for: test-principal 3: I1211 23:37:31.627847 16011 credentials.hpp:37] Loading credentials for authentication from '/tmp/MasterSlaveReconciliationTest_ReconcileDroppedTask_wU5QUB/http_credentials' 3: I1211 23:37:31.628092 16011 http.cpp:1045] Creating default 'basic' HTTP authenticator for realm 'mesos-agent-readonly' 3: I1211 23:37:31.628286 16011 http.cpp:1045] Creating default 'basic' HTTP authenticator for realm 'mesos-agent-readwrite' 3: I1211 23:37:31.628703 15995 sched.cpp:232] Version: 1.5.0 3: I1211 23:37:31.629477 16002 sched.cpp:336] New master detected at [email protected]:50774 3: I1211 23:37:31.629582 16002 sched.cpp:396] Authenticating with master [email protected]:50774 3: I1211 23:37:31.629601 16002 sched.cpp:403] Using default CRAM-MD5 authenticatee 3: I1211 23:37:31.629884 16006 authenticatee.cpp:121] Creating new client SASL connection 3: I1211 23:37:31.630225 16003 master.cpp:8888] Authenticating [email protected]:50774 3: I1211 23:37:31.630383 16010 authenticator.cpp:414] Starting authentication session for crammd5-authenticatee(210)@172.17.0.5:50774 3: I1211 23:37:31.630372 16011 slave.cpp:590] Agent resources: [{"name":"cpus","scalar":{"value":2.0},"type":"SCALAR"},{"name":"mem","scalar":{"value":1024.0},"type":"SCALAR"},{"name":"disk","scalar":{"value":1024.0},"type":"SCALAR"},{"name":"ports","ranges":{"range":[{"begin":31000,"end":32000}]},"type":"RANGES"}] 3: I1211 23:37:31.630645 16011 slave.cpp:598] Agent attributes: [ ] 3: I1211 23:37:31.630655 16011 slave.cpp:607] Agent hostname: da72a4ac167e 3: I1211 23:37:31.630661 16004 authenticator.cpp:98] Creating new server SASL connection 3: I1211 23:37:31.630789 15996 task_status_update_manager.cpp:181] Pausing sending task status updates 3: I1211 23:37:31.630913 16000 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5 3: I1211 23:37:31.630950 16000 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5' 3: I1211 23:37:31.631068 15997 authenticator.cpp:204] Received SASL authentication start 3: I1211 23:37:31.631132 15997 authenticator.cpp:326] Authentication requires more steps 3: I1211 23:37:31.631250 16007 authenticatee.cpp:259] Received SASL authentication step 3: I1211 23:37:31.631377 16007 authenticator.cpp:232] Received SASL authentication step 3: I1211 23:37:31.631407 16007 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: 'da72a4ac167e' server FQDN: 'da72a4ac167e' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false 3: I1211 23:37:31.631422 16007 auxprop.cpp:181] Looking up auxiliary property '*userPassword' 3: I1211 23:37:31.631453 16007 auxprop.cpp:181] Looking up auxiliary property '*cmusaslsecretCRAM-MD5' 3: I1211 23:37:31.631475 16007 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: 'da72a4ac167e' server FQDN: 'da72a4ac167e' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true 3: I1211 23:37:31.631485 16007 auxprop.cpp:131] Skipping auxiliary property '*userPassword' since SASL_AUXPROP_AUTHZID == true 3: I1211 23:37:31.631490 16007 auxprop.cpp:131] Skipping auxiliary property '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true 3: I1211 23:37:31.631503 16007 authenticator.cpp:318] Authentication success 3: I1211 23:37:31.631680 15999 master.cpp:8918] Successfully authenticated principal 'test-principal' at [email protected]:50774 3: I1211 23:37:31.631716 16008 authenticatee.cpp:299] Authentication success 3: I1211 23:37:31.631757 16009 authenticator.cpp:432] Authentication session cleanup for crammd5-authenticatee(210)@172.17.0.5:50774 3: I1211 23:37:31.632030 16002 sched.cpp:502] Successfully authenticated with master [email protected]:50774 3: I1211 23:37:31.632046 16002 sched.cpp:824] Sending SUBSCRIBE call to [email protected]:50774 3: I1211 23:37:31.632153 16002 sched.cpp:857] Will retry registration in 818.979043ms if necessary 3: I1211 23:37:31.632345 15996 state.cpp:66] Recovering state from '/tmp/MasterSlaveReconciliationTest_ReconcileDroppedTask_dNGXl0/meta' 3: I1211 23:37:31.632352 16003 master.cpp:2957] Received SUBSCRIBE call for framework 'default' at [email protected]:50774 3: I1211 23:37:31.632372 16003 master.cpp:2274] Authorizing framework principal 'test-principal' to receive offers for roles '{ * }' 3: I1211 23:37:31.632702 15997 task_status_update_manager.cpp:207] Recovering task status update manager 3: I1211 23:37:31.632889 15999 containerizer.cpp:674] Recovering containerizer 3: I1211 23:37:31.632941 16008 master.cpp:3037] Subscribing framework default with checkpointing disabled and capabilities [ MULTI_ROLE, RESERVATION_REFINEMENT, PARTITION_AWARE ] 3: I1211 23:37:31.633142 16008 master.cpp:9109] Adding framework 156d40d2-e105-4862-ba03-3fdd2fe950ce-0000 (default) at [email protected]:50774 with roles { } suppressed 3: I1211 23:37:31.633610 15998 sched.cpp:751] Framework registered with 156d40d2-e105-4862-ba03-3fdd2fe950ce-0000 3: I1211 23:37:31.633677 15998 sched.cpp:765] Scheduler::registered took 48710ns 3: I1211 23:37:31.633730 16006 hierarchical.cpp:297] Added framework 156d40d2-e105-4862-ba03-3fdd2fe950ce-0000 3: I1211 23:37:31.633919 16006 hierarchical.cpp:1513] Performed allocation for 0 agents in 75044ns 3: I1211 23:37:31.634531 16002 provisioner.cpp:495] Provisioner recovery complete 3: I1211 23:37:31.634824 16008 slave.cpp:6727] Finished recovery 3: I1211 23:37:31.635624 15999 slave.cpp:1033] New master detected at [email protected]:50774 3: I1211 23:37:31.635644 16010 task_status_update_manager.cpp:181] Pausing sending task status updates 3: I1211 23:37:31.635740 15999 slave.cpp:1088] Detecting new master 3: I1211 23:37:31.639170 15998 slave.cpp:1115] Authenticating with master [email protected]:50774 3: I1211 23:37:31.639238 15998 slave.cpp:1124] Using default CRAM-MD5 authenticatee 3: I1211 23:37:31.639477 16000 authenticatee.cpp:121] Creating new client SASL connection 3: I1211 23:37:31.639819 16006 master.cpp:8888] Authenticating slave(101)@172.17.0.5:50774 3: I1211 23:37:31.639982 16009 authenticator.cpp:414] Starting authentication session for crammd5-authenticatee(211)@172.17.0.5:50774 3: I1211 23:37:31.640224 16007 authenticator.cpp:98] Creating new server SASL connection 3: I1211 23:37:31.640424 15997 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5 3: I1211 23:37:31.640444 15997 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5' 3: I1211 23:37:31.640538 16004 authenticator.cpp:204] Received SASL authentication start 3: I1211 23:37:31.640585 16004 authenticator.cpp:326] Authentication requires more steps 3: I1211 23:37:31.640692 16005 authenticatee.cpp:259] Received SASL authentication step 3: I1211 23:37:31.640802 16002 authenticator.cpp:232] Received SASL authentication step 3: I1211 23:37:31.640827 16002 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: 'da72a4ac167e' server FQDN: 'da72a4ac167e' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false 3: I1211 23:37:31.640838 16002 auxprop.cpp:181] Looking up auxiliary property '*userPassword' 3: I1211 23:37:31.640862 16002 auxprop.cpp:181] Looking up auxiliary property '*cmusaslsecretCRAM-MD5' 3: I1211 23:37:31.640877 16002 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: 'da72a4ac167e' server FQDN: 'da72a4ac167e' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true 3: I1211 23:37:31.640890 16002 auxprop.cpp:131] Skipping auxiliary property '*userPassword' since SASL_AUXPROP_AUTHZID == true 3: I1211 23:37:31.640895 16002 auxprop.cpp:131] Skipping auxiliary property '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true 3: I1211 23:37:31.640908 16002 authenticator.cpp:318] Authentication success 3: I1211 23:37:31.640985 16003 authenticatee.cpp:299] Authentication success 3: I1211 23:37:31.641041 16008 master.cpp:8918] Successfully authenticated principal 'test-principal' at slave(101)@172.17.0.5:50774 3: I1211 23:37:31.641111 15996 authenticator.cpp:432] Authentication session cleanup for crammd5-authenticatee(211)@172.17.0.5:50774 3: I1211 23:37:31.641286 16001 slave.cpp:1207] Successfully authenticated with master [email protected]:50774 3: I1211 23:37:31.641496 16001 slave.cpp:1684] Will retry registration in 5.721782ms if necessary 3: I1211 23:37:31.641630 16000 master.cpp:6077] Received register agent message from slave(101)@172.17.0.5:50774 (da72a4ac167e) 3: I1211 23:37:31.641669 16000 master.cpp:3866] Authorizing agent with principal 'test-principal' 3: I1211 23:37:31.641989 16009 master.cpp:6147] Authorized registration of agent at slave(101)@172.17.0.5:50774 (da72a4ac167e) 3: I1211 23:37:31.642062 16009 master.cpp:6240] Registering agent at slave(101)@172.17.0.5:50774 (da72a4ac167e) with id 156d40d2-e105-4862-ba03-3fdd2fe950ce-S0 Build step 'Execute shell' marked build as failure Not sending mail to unregistered user [email protected]
