See <https://builds.apache.org/job/Mesos-Buildbot/BUILDTOOL=cmake,COMPILER=gcc,CONFIGURATION=--verbose%20--disable-libtool-wrappers,ENVIRONMENT=GLOG_v=1%20MESOS_VERBOSE=1,OS=ubuntu%3A14.04,label_exp=(ubuntu)&&(!ubuntu-us1)&&(!ubuntu-eu2)&&(!qnode3)&&(!H23)/4522/display/redirect?page=changes>
Changes: [yujie.jay] Removed some dead code in the master. [andrew] Windows: Added `os::set_job_cpu_limit` to stout. [andrew] Windows: Added `os::set_job_mem_limit` to stout. [andrew] Windows: Added `os::get_job_info` to stout. [andrew] Windows: Added `os::get_job_processes` to stout. [andrew] Windows: Added `os::get_job_mem` to stout. [andrew] Windows: Abstracted out `os::name_job` in stout. [andrew] Windows: Added `Cpu` and `Mem` isolators. [andrew] Windows: Ported `Cpu` and `Mem` isolator tests. [andrew] Windows: Documented the `cpu` and `mem` isolators. [andrew] Windows: Fixed `SlaveReregisterTaskExecutorIds` test. ------------------------------------------ [...truncated 5.49 MB...] 3: I1201 01:27:46.862982 17334 hierarchical.cpp:586] Removed agent 0578fa53-fea6-4e8c-9421-6f8075300357-S0 3: I1201 01:27:46.862987 17330 master.cpp:9913] Removing task 31c44447-55ba-4ba1-8aa5-537f78a872f7 with resources cpus(allocated: *):2; mem(allocated: *):1024; disk(allocated: *):1024; ports(allocated: *):[31000-32000] of framework 0578fa53-fea6-4e8c-9421-6f8075300357-0000 on agent 0578fa53-fea6-4e8c-9421-6f8075300357-S0 at slave(109)@172.17.0.4:52315 (3e44a50127b8) 3: I1201 01:27:46.863281 17330 master.cpp:7608] Sending status update TASK_LOST for task 31c44447-55ba-4ba1-8aa5-537f78a872f7 of framework 0578fa53-fea6-4e8c-9421-6f8075300357-0000 'health check timed out' 3: I1201 01:27:46.878130 17330 master.cpp:2138] Notifying framework 0578fa53-fea6-4e8c-9421-6f8075300357-0000 (default) at [email protected]:52315 of lost agent 0578fa53-fea6-4e8c-9421-6f8075300357-S0 (3e44a50127b8) 3: I1201 01:27:46.878244 17334 sched.cpp:1029] Scheduler::statusUpdate took 97368ns 3: I1201 01:27:46.878530 17334 sched.cpp:1091] Lost agent 0578fa53-fea6-4e8c-9421-6f8075300357-S0 3: I1201 01:27:46.878607 17334 sched.cpp:1102] Scheduler::slaveLost took 33363ns 3: I1201 01:27:46.883891 17338 process.cpp:3503] Handling HTTP event for process 'metrics' with path: '/metrics/snapshot' 3: I1201 01:27:46.885184 17338 http.cpp:851] Authorizing principal 'ANY' to GET the endpoint '/metrics/snapshot' 3: I1201 01:27:46.961885 17337 process.cpp:3503] Handling HTTP event for process 'master' with path: '/master/state' 3: I1201 01:27:46.964293 17337 http.cpp:1185] HTTP GET for /master/state from 172.17.0.4:35427 3: I1201 01:27:46.999550 17334 slave.cpp:992] Lost leading master 3: I1201 01:27:46.999585 17334 slave.cpp:1034] Detecting new master 3: I1201 01:27:46.999717 17334 task_status_update_manager.cpp:181] Pausing sending task status updates 3: I1201 01:27:47.000094 17334 slave.cpp:999] New master detected at [email protected]:52315 3: I1201 01:27:47.000250 17334 slave.cpp:1034] Detecting new master 3: I1201 01:27:47.000382 17334 task_status_update_manager.cpp:181] Pausing sending task status updates 3: I1201 01:27:47.001621 17333 slave.cpp:1061] Authenticating with master [email protected]:52315 3: I1201 01:27:47.001716 17333 slave.cpp:1070] Using default CRAM-MD5 authenticatee 3: I1201 01:27:47.002135 17333 authenticatee.cpp:121] Creating new client SASL connection 3: I1201 01:27:47.002557 17333 master.cpp:8589] Authenticating slave(109)@172.17.0.4:52315 3: I1201 01:27:47.002760 17333 authenticator.cpp:414] Starting authentication session for crammd5-authenticatee(237)@172.17.0.4:52315 3: I1201 01:27:47.003180 17333 authenticator.cpp:98] Creating new server SASL connection 3: I1201 01:27:47.003453 17333 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5 3: I1201 01:27:47.003476 17333 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5' 3: I1201 01:27:47.003588 17333 authenticator.cpp:204] Received SASL authentication start 3: I1201 01:27:47.003644 17333 authenticator.cpp:326] Authentication requires more steps 3: I1201 01:27:47.003752 17333 authenticatee.cpp:259] Received SASL authentication step 3: I1201 01:27:47.003868 17333 authenticator.cpp:232] Received SASL authentication step 3: I1201 01:27:47.003891 17333 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: '3e44a50127b8' server FQDN: '3e44a50127b8' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false 3: I1201 01:27:47.003902 17333 auxprop.cpp:181] Looking up auxiliary property '*userPassword' 3: I1201 01:27:47.003947 17333 auxprop.cpp:181] Looking up auxiliary property '*cmusaslsecretCRAM-MD5' 3: I1201 01:27:47.003964 17333 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: '3e44a50127b8' server FQDN: '3e44a50127b8' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true 3: I1201 01:27:47.003974 17333 auxprop.cpp:131] Skipping auxiliary property '*userPassword' since SASL_AUXPROP_AUTHZID == true 3: I1201 01:27:47.003983 17333 auxprop.cpp:131] Skipping auxiliary property '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true 3: I1201 01:27:47.003998 17333 authenticator.cpp:318] Authentication success 3: I1201 01:27:47.004215 17333 authenticatee.cpp:299] Authentication success 3: I1201 01:27:47.004313 17333 master.cpp:8619] Successfully authenticated principal 'test-principal' at slave(109)@172.17.0.4:52315 3: I1201 01:27:47.004401 17333 authenticator.cpp:432] Authentication session cleanup for crammd5-authenticatee(237)@172.17.0.4:52315 3: I1201 01:27:47.004765 17333 slave.cpp:1153] Successfully authenticated with master [email protected]:52315 3: I1201 01:27:47.006140 17325 master.cpp:6371] Received re-register agent message from agent 0578fa53-fea6-4e8c-9421-6f8075300357-S0 at slave(109)@172.17.0.4:52315 (3e44a50127b8) 3: I1201 01:27:47.006397 17325 master.cpp:3878] Authorizing agent with principal 'test-principal' 3: I1201 01:27:47.007091 17325 master.cpp:6442] Authorized re-registration of agent 0578fa53-fea6-4e8c-9421-6f8075300357-S0 at slave(109)@172.17.0.4:52315 (3e44a50127b8) 3: I1201 01:27:47.007182 17325 master.cpp:6624] Re-registering agent 0578fa53-fea6-4e8c-9421-6f8075300357-S0 at slave(109)@172.17.0.4:52315 (3e44a50127b8) 3: I1201 01:27:47.007851 17325 registrar.cpp:495] Applied 1 operations in 90367ns; attempting to update the registry 3: I1201 01:27:47.008803 17325 registrar.cpp:552] Successfully updated the registry in 0ns 3: I1201 01:27:47.009212 17339 master.cpp:6696] Re-admitted agent 0578fa53-fea6-4e8c-9421-6f8075300357-S0 at slave(109)@172.17.0.4:52315 (3e44a50127b8) 3: I1201 01:27:47.009932 17339 master.cpp:11130] Adding task 31c44447-55ba-4ba1-8aa5-537f78a872f7 with resources cpus(allocated: *):2; mem(allocated: *):1024; disk(allocated: *):1024; ports(allocated: *):[31000-32000] on agent 0578fa53-fea6-4e8c-9421-6f8075300357-S0 at slave(109)@172.17.0.4:52315 (3e44a50127b8) 3: I1201 01:27:47.010664 17339 master.cpp:6849] Re-registered agent 0578fa53-fea6-4e8c-9421-6f8075300357-S0 at slave(109)@172.17.0.4:52315 (3e44a50127b8) with cpus:2; mem:1024; disk:1024; ports:[31000-32000] 3: I1201 01:27:47.011965 17339 hierarchical.cpp:553] Added agent 0578fa53-fea6-4e8c-9421-6f8075300357-S0 (3e44a50127b8) with cpus:2; mem:1024; disk:1024; ports:[31000-32000] (allocated: cpus(allocated: *):2; mem(allocated: *):1024; disk(allocated: *):1024; ports(allocated: *):[31000-32000]) 3: I1201 01:27:47.012315 17339 hierarchical.cpp:1431] Performed allocation for 1 agents in 167115ns 3: I1201 01:27:47.012516 17333 slave.cpp:1696] Will retry registration in 17.239296ms if necessary 3: I1201 01:27:47.012816 17333 slave.cpp:1343] Re-registered with master [email protected]:52315 3: I1201 01:27:47.012985 17333 slave.cpp:1411] Forwarding total oversubscribed resources {} 3: I1201 01:27:47.013157 17332 task_status_update_manager.cpp:188] Resuming sending task status updates 3: I1201 01:27:47.013360 17332 master.cpp:7049] Received update of agent 0578fa53-fea6-4e8c-9421-6f8075300357-S0 at slave(109)@172.17.0.4:52315 (3e44a50127b8) with total oversubscribed resources {} 3: I1201 01:27:47.013572 17332 master.cpp:7092] Ignoring update on agent 0578fa53-fea6-4e8c-9421-6f8075300357-S0 at slave(109)@172.17.0.4:52315 (3e44a50127b8) as it reports no changes 3: I1201 01:27:47.013794 17333 slave.cpp:3452] Updating info for framework 0578fa53-fea6-4e8c-9421-6f8075300357-0000 with pid updated to [email protected]:52315 3: I1201 01:27:47.013913 17328 task_status_update_manager.cpp:188] Resuming sending task status updates 3: I1201 01:27:47.014667 17331 master.cpp:8073] Performing explicit task state reconciliation for 1 tasks of framework 0578fa53-fea6-4e8c-9421-6f8075300357-0000 (default) at [email protected]:52315 3: I1201 01:27:47.014806 17331 master.cpp:8221] Sending explicit reconciliation state TASK_RUNNING for task 31c44447-55ba-4ba1-8aa5-537f78a872f7 of framework 0578fa53-fea6-4e8c-9421-6f8075300357-0000 (default) at [email protected]:52315 3: I1201 01:27:47.015224 17331 sched.cpp:1029] Scheduler::statusUpdate took 69468ns 3: I1201 01:27:47.018959 17337 process.cpp:3503] Handling HTTP event for process 'master' with path: '/master/state' 3: I1201 01:27:47.020814 17335 http.cpp:1185] HTTP GET for /master/state from 172.17.0.4:35428 3: I1201 01:27:47.051713 17333 process.cpp:3503] Handling HTTP event for process 'master' with path: '/master/state-summary' 3: I1201 01:27:47.053431 17337 http.cpp:1185] HTTP GET for /master/state-summary from 172.17.0.4:35429 3: I1201 01:27:47.085711 17325 process.cpp:3503] Handling HTTP event for process 'metrics' with path: '/metrics/snapshot' 3: I1201 01:27:47.086828 17325 http.cpp:851] Authorizing principal 'ANY' to GET the endpoint '/metrics/snapshot' 3: I1201 01:27:47.112888 17323 sched.cpp:2009] Asked to stop the driver 3: I1201 01:27:47.117856 17330 sched.cpp:1191] Stopping framework 0578fa53-fea6-4e8c-9421-6f8075300357-0000 3: I1201 01:27:47.118293 17339 master.cpp:9096] Processing TEARDOWN call for framework 0578fa53-fea6-4e8c-9421-6f8075300357-0000 (default) at [email protected]:52315 3: I1201 01:27:47.118386 17339 master.cpp:9108] Removing framework 0578fa53-fea6-4e8c-9421-6f8075300357-0000 (default) at [email protected]:52315 3: I1201 01:27:47.118435 17339 master.cpp:3339] Deactivating framework 0578fa53-fea6-4e8c-9421-6f8075300357-0000 (default) at [email protected]:52315 3: I1201 01:27:47.118608 17328 hierarchical.cpp:406] Deactivated framework 0578fa53-fea6-4e8c-9421-6f8075300357-0000 3: I1201 01:27:47.118831 17337 slave.cpp:3281] Asked to shut down framework 0578fa53-fea6-4e8c-9421-6f8075300357-0000 by [email protected]:52315 3: I1201 01:27:47.118850 17339 master.cpp:9807] Updating the state of task 31c44447-55ba-4ba1-8aa5-537f78a872f7 of framework 0578fa53-fea6-4e8c-9421-6f8075300357-0000 (latest state: TASK_KILLED, status update state: TASK_KILLED) 3: I1201 01:27:47.119788 17339 master.cpp:9913] Removing task 31c44447-55ba-4ba1-8aa5-537f78a872f7 with resources cpus(allocated: *):2; mem(allocated: *):1024; disk(allocated: *):1024; ports(allocated: *):[31000-32000] of framework 0578fa53-fea6-4e8c-9421-6f8075300357-0000 on agent 0578fa53-fea6-4e8c-9421-6f8075300357-S0 at slave(109)@172.17.0.4:52315 (3e44a50127b8) 3: I1201 01:27:47.118930 17337 slave.cpp:3306] Shutting down framework 0578fa53-fea6-4e8c-9421-6f8075300357-0000 3: I1201 01:27:47.121209 17337 slave.cpp:5946] Shutting down executor '31c44447-55ba-4ba1-8aa5-537f78a872f7' of framework 0578fa53-fea6-4e8c-9421-6f8075300357-0000 at executor(1)@172.17.0.4:50306 3: I1201 01:27:47.121417 17339 hierarchical.cpp:1106] Recovered cpus(allocated: *):2; mem(allocated: *):1024; disk(allocated: *):1024; ports(allocated: *):[31000-32000] (total: cpus:2; mem:1024; disk:1024; ports:[31000-32000], allocated: {}) on agent 0578fa53-fea6-4e8c-9421-6f8075300357-S0 from framework 0578fa53-fea6-4e8c-9421-6f8075300357-0000 3: I1201 01:27:47.121717 17339 hierarchical.cpp:345] Removed framework 0578fa53-fea6-4e8c-9421-6f8075300357-0000 3: I1201 01:27:47.126389 18859 exec.cpp:435] Executor asked to shutdown 3: I1201 01:27:47.127284 18859 executor.cpp:171] Received SHUTDOWN event 3: I1201 01:27:47.127307 18859 executor.cpp:747] Shutting down 3: I1201 01:27:47.127358 18859 executor.cpp:854] Sending SIGTERM to process tree at pid 18864 3: I1201 01:27:47.135218 18859 executor.cpp:867] Sent SIGTERM to the following process trees: 3: [ 3: -+- 18864 sh -c sleep 60 3: \--- 18865 sleep 60 3: ] 3: I1201 01:27:47.135287 18859 executor.cpp:871] Scheduling escalation to SIGKILL in 3secs from now 3: I1201 01:27:47.136274 17338 slave.cpp:875] Agent terminating 3: I1201 01:27:47.136333 18859 executor.cpp:929] Command terminated with signal Terminated (pid: 18864) 3: I1201 01:27:47.136448 17338 slave.cpp:3281] Asked to shut down framework 0578fa53-fea6-4e8c-9421-6f8075300357-0000 by @0.0.0.0:0 3: W1201 01:27:47.136485 17338 slave.cpp:3302] Ignoring shutdown framework 0578fa53-fea6-4e8c-9421-6f8075300357-0000 because it is terminating 3: I1201 01:27:47.153478 17338 master.cpp:1317] Agent 0578fa53-fea6-4e8c-9421-6f8075300357-S0 at slave(109)@172.17.0.4:52315 (3e44a50127b8) disconnected 3: I1201 01:27:47.153664 17338 master.cpp:3376] Disconnecting agent 0578fa53-fea6-4e8c-9421-6f8075300357-S0 at slave(109)@172.17.0.4:52315 (3e44a50127b8) 3: I1201 01:27:47.153794 17338 master.cpp:3395] Deactivating agent 0578fa53-fea6-4e8c-9421-6f8075300357-S0 at slave(109)@172.17.0.4:52315 (3e44a50127b8) 3: I1201 01:27:47.154152 17338 hierarchical.cpp:671] Agent 0578fa53-fea6-4e8c-9421-6f8075300357-S0 deactivated 3: I1201 01:27:47.157483 17340 process.cpp:2578] Failed to deliver libprocess message to /slave(109)/mesos.internal.StatusUpdateMessage 3: I1201 01:27:47.158141 17328 containerizer.cpp:2326] Destroying container fbda24ea-9d50-4e1c-8bcc-4914eea91ec0 in RUNNING state 3: I1201 01:27:47.158170 17328 containerizer.cpp:2923] Transitioning the state of container fbda24ea-9d50-4e1c-8bcc-4914eea91ec0 from RUNNING to DESTROYING 3: I1201 01:27:47.158530 17328 launcher.cpp:156] Asked to destroy container fbda24ea-9d50-4e1c-8bcc-4914eea91ec0 3: I1201 01:27:47.165221 17331 hierarchical.cpp:1431] Performed allocation for 1 agents in 204409ns 3: I1201 01:27:47.173406 17340 process.cpp:774] Decode failure: Connection reset by peer 3: I1201 01:27:47.173559 17340 process.cpp:2139] Failed to shutdown socket with fd 7, address 172.17.0.4:52315: Transport endpoint is not connected 3: I1201 01:27:47.226439 17338 containerizer.cpp:2777] Container fbda24ea-9d50-4e1c-8bcc-4914eea91ec0 has exited 3: I1201 01:27:47.230406 17325 provisioner.cpp:595] Ignoring destroy request for unknown container fbda24ea-9d50-4e1c-8bcc-4914eea91ec0 3: I1201 01:27:47.345515 17323 master.cpp:1159] Master terminating 3: I1201 01:27:47.346994 17327 hierarchical.cpp:586] Removed agent 0578fa53-fea6-4e8c-9421-6f8075300357-S0 3: [ OK ] PartitionTest.ReregisterSlaveNotPartitionAware (1099 ms) 3: [ RUN ] PartitionTest.PartitionedSlaveReregistrationMasterFailover 3: I1201 01:27:47.364881 17323 cluster.cpp:170] Creating default 'local' authorizer 3: I1201 01:27:47.379663 17328 master.cpp:454] Master d7ac1d4c-38ff-4a0c-9c6c-44a867cdac57 (3e44a50127b8) started on 172.17.0.4:52315 3: I1201 01:27:47.379698 17328 master.cpp:456] Flags at startup: --acls="" --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins" --allocation_interval="1secs" --allocator="HierarchicalDRF" --authenticate_agents="true" --authenticate_frameworks="true" --authenticate_http_frameworks="true" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticators="crammd5" --authorizers="local" --credentials="/tmp/186RgC/credentials" --filter_gpu_resources="true" --framework_sorter="drf" --help="false" --hostname_lookup="true" --http_authenticators="basic" --http_framework_authenticators="basic" --initialize_driver_logging="true" --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO" --max_agent_ping_timeouts="5" --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000" --max_unreachable_tasks_per_framework="1000" --port="5050" --quiet="false" --recovery_agent_removal_limit="100%" --registry="in_memory" --registry_fetch_timeout="1mins" --registry_gc_interval="15mins" --registry_max_agent_age="2weeks" --registry_max_agent_count="102400" --registry_store_timeout="100secs" --registry_strict="false" --root_submissions="true" --user_sorter="drf" --version="false" --webui_dir="/usr/local/share/mesos/webui" --work_dir="/tmp/186RgC/master" --zk_session_timeout="10secs" 3: I1201 01:27:47.380182 17328 master.cpp:505] Master only allowing authenticated frameworks to register 3: I1201 01:27:47.380190 17328 master.cpp:511] Master only allowing authenticated agents to register 3: I1201 01:27:47.380197 17328 master.cpp:517] Master only allowing authenticated HTTP frameworks to register 3: I1201 01:27:47.380203 17328 credentials.hpp:37] Loading credentials for authentication from '/tmp/186RgC/credentials' 3: I1201 01:27:47.380548 17328 master.cpp:561] Using default 'crammd5' authenticator 3: I1201 01:27:47.380774 17328 http.cpp:1045] Creating default 'basic' HTTP authenticator for realm 'mesos-master-readonly' 3: I1201 01:27:47.380992 17328 http.cpp:1045] Creating default 'basic' HTTP authenticator for realm 'mesos-master-readwrite' 3: I1201 01:27:47.381736 17328 http.cpp:1045] Creating default 'basic' HTTP authenticator for realm 'mesos-master-scheduler' 3: I1201 01:27:47.382218 17328 master.cpp:640] Authorization enabled 3: I1201 01:27:47.382413 17339 hierarchical.cpp:173] Initialized hierarchical allocator process 3: I1201 01:27:47.382784 17330 whitelist_watcher.cpp:77] No whitelist given 3: I1201 01:27:47.389456 17328 master.cpp:2221] Elected as the leading master! 3: I1201 01:27:47.389492 17328 master.cpp:1701] Recovering from registrar 3: I1201 01:27:47.389628 17327 registrar.cpp:347] Recovering registrar 3: I1201 01:27:47.390573 17327 registrar.cpp:391] Successfully fetched the registry (0B) in 0ns 3: I1201 01:27:47.390696 17327 registrar.cpp:495] Applied 1 operations in 43272ns; attempting to update the registry 3: I1201 01:27:47.391513 17327 registrar.cpp:552] Successfully updated the registry in 0ns 3: I1201 01:27:47.391631 17327 registrar.cpp:424] Successfully recovered registrar 3: I1201 01:27:47.392223 17328 master.cpp:1814] Recovered 0 agents from the registry (129B); allowing 10mins for agents to re-register 3: I1201 01:27:47.392459 17328 hierarchical.cpp:211] Skipping recovery of hierarchical allocator: nothing to recover 3: W1201 01:27:47.421350 17323 process.cpp:2756] Attempted to spawn already running process [email protected]:52315 3: I1201 01:27:47.422744 17323 containerizer.cpp:304] Using isolation { environment_secret, posix/cpu, posix/mem, filesystem/posix, network/cni } 3: W1201 01:27:47.423473 17323 backend.cpp:76] Failed to create 'aufs' backend: AufsBackend requires root privileges 3: W1201 01:27:47.423635 17323 backend.cpp:76] Failed to create 'bind' backend: BindBackend requires root privileges 3: I1201 01:27:47.423668 17323 provisioner.cpp:297] Using default backend 'copy' 3: I1201 01:27:47.426136 17323 cluster.cpp:458] Creating default 'local' authorizer 3: I1201 01:27:47.431802 17336 slave.cpp:253] Mesos agent started on (110)@172.17.0.4:52315 3: I1201 01:27:47.431843 17336 slave.cpp:254] Flags at startup: --acls="" --appc_simple_discovery_uri_prefix="http://"; --appc_store_dir="/tmp/PartitionTest_PartitionedSlaveReregistrationMasterFailover_p9X21E/store/appc" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs" --containerizers="mesos" --credential="/tmp/PartitionTest_PartitionedSlaveReregistrationMasterFailover_p9X21E/credential" --default_role="*" --disallow_sharing_agent_pid_namespace="false" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io"; --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/tmp/PartitionTest_PartitionedSlaveReregistrationMasterFailover_p9X21E/store/docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_reregistration_timeout="2secs" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/PartitionTest_PartitionedSlaveReregistrationMasterFailover_p9X21E/fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname_lookup="true" --http_command_executor="false" --http_credentials="/tmp/PartitionTest_PartitionedSlaveReregistrationMasterFailover_p9X21E/http_credentials" --http_heartbeat_interval="30secs" --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" --launcher="posix" --launcher_dir="/mesos/build/src" --logbufsecs="0" --logging_level="INFO" --max_completed_executors_per_framework="150" --oversubscribed_resources_interval="15secs" --perf_duration="10secs" --perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="10ms" --resources="cpus:2;mem:1024" --revocable_cpu_low_priority="true" --runtime_dir="/tmp/PartitionTest_PartitionedSlaveReregistrationMasterFailover_p9X21E" --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" --systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/system" --version="false" --work_dir="/tmp/PartitionTest_PartitionedSlaveReregistrationMasterFailover_bxc0C8" --zk_session_timeout="10secs" 3: I1201 01:27:47.432538 17336 credentials.hpp:86] Loading credential for authentication from '/tmp/PartitionTest_PartitionedSlaveReregistrationMasterFailover_p9X21E/credential' 3: I1201 01:27:47.432835 17336 slave.cpp:286] Agent using credential for: test-principal 3: I1201 01:27:47.432853 17336 credentials.hpp:37] Loading credentials for authentication from '/tmp/PartitionTest_PartitionedSlaveReregistrationMasterFailover_p9X21E/http_credentials' 3: I1201 01:27:47.433195 17336 http.cpp:1045] Creating default 'basic' HTTP authenticator for realm 'mesos-agent-readonly' 3: I1201 01:27:47.433531 17336 http.cpp:1045] Creating default 'basic' HTTP authenticator for realm 'mesos-agent-readwrite' 3: W1201 01:27:47.433954 17323 process.cpp:2756] Attempted to spawn already running process [email protected]:52315 3: I1201 01:27:47.435137 17323 sched.cpp:232] Version: 1.5.0 3: I1201 01:27:47.435354 17336 slave.cpp:585] Agent resources: [{"name":"cpus","scalar":{"value":2.0},"type":"SCALAR"},{"name":"mem","scalar":{"value":1024.0},"type":"SCALAR"},{"name":"disk","scalar":{"value":3701220.0},"type":"SCALAR"},{"name":"ports","ranges":{"range":[{"begin":31000,"end":32000}]},"type":"RANGES"}] 3: I1201 01:27:47.435708 17336 slave.cpp:593] Agent attributes: [ ] 3: I1201 01:27:47.435722 17336 slave.cpp:602] Agent hostname: 3e44a50127b8 3: I1201 01:27:47.435953 17339 task_status_update_manager.cpp:181] Pausing sending task status updates 3: I1201 01:27:47.436012 17339 sched.cpp:336] New master detected at [email protected]:52315 3: I1201 01:27:47.436154 17339 sched.cpp:396] Authenticating with master [email protected]:52315 3: I1201 01:27:47.436167 17339 sched.cpp:403] Using default CRAM-MD5 authenticatee 3: I1201 01:27:47.436606 17339 authenticatee.cpp:121] Creating new client SASL connection 3: I1201 01:27:47.437340 17329 master.cpp:8589] Authenticating [email protected]:52315 3: I1201 01:27:47.437492 17334 authenticator.cpp:414] Starting authentication session for crammd5-authenticatee(238)@172.17.0.4:52315 3: I1201 01:27:47.437973 17334 authenticator.cpp:98] Creating new server SASL connection 3: I1201 01:27:47.438283 17334 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5 3: I1201 01:27:47.438305 17334 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5' 3: I1201 01:27:47.438424 17334 authenticator.cpp:204] Received SASL authentication start 3: I1201 01:27:47.438479 17334 authenticator.cpp:326] Authentication requires more steps 3: I1201 01:27:47.438596 17334 authenticatee.cpp:259] Received SASL authentication step 3: I1201 01:27:47.438720 17334 authenticator.cpp:232] Received SASL authentication step 3: I1201 01:27:47.438748 17334 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: '3e44a50127b8' server FQDN: '3e44a50127b8' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false 3: I1201 01:27:47.438760 17334 auxprop.cpp:181] Looking up auxiliary property '*userPassword' 3: I1201 01:27:47.438802 17334 auxprop.cpp:181] Looking up auxiliary property '*cmusaslsecretCRAM-MD5' 3: I1201 01:27:47.438823 17334 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: '3e44a50127b8' server FQDN: '3e44a50127b8' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true 3: I1201 01:27:47.438833 17334 auxprop.cpp:131] Skipping auxiliary property '*userPassword' since SASL_AUXPROP_AUTHZID == true 3: I1201 01:27:47.438841 17334 auxprop.cpp:131] Skipping auxiliary property '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true 3: I1201 01:27:47.438858 17334 authenticator.cpp:318] Authentication success 3: I1201 01:27:47.439040 17334 authenticatee.cpp:299] Authentication success 3: I1201 01:27:47.439155 17334 authenticator.cpp:432] Authentication session cleanup for crammd5-authenticatee(238)@172.17.0.4:52315 3: I1201 01:27:47.439534 17334 sched.cpp:502] Successfully authenticated with master [email protected]:52315 3: I1201 01:27:47.439550 17334 sched.cpp:824] Sending SUBSCRIBE call to [email protected]:52315 3: I1201 01:27:47.439666 17334 sched.cpp:857] Will retry registration in 1.624128516secs if necessary 3: I1201 01:27:47.440030 17329 master.cpp:2871] Queuing up SUBSCRIBE call for framework 'default' at [email protected]:52315 because authentication is still in progress 3: I1201 01:27:47.440214 17329 master.cpp:8619] Successfully authenticated principal 'test-principal' at [email protected]:52315 3: I1201 01:27:47.440433 17329 master.cpp:2969] Received SUBSCRIBE call for framework 'default' at [email protected]:52315 3: I1201 01:27:47.440552 17329 master.cpp:2286] Authorizing framework principal 'test-principal' to receive offers for roles '{ * }' 3: I1201 01:27:47.441154 17334 master.cpp:3049] Subscribing framework default with checkpointing disabled and capabilities [ RESERVATION_REFINEMENT ] 3: I1201 01:27:47.441812 17327 hierarchical.cpp:1431] Performed allocation for 0 agents in 64593ns 3: I1201 01:27:47.442653 17334 sched.cpp:751] Framework registered with d7ac1d4c-38ff-4a0c-9c6c-44a867cdac57-0000 3: I1201 01:27:47.442706 17334 sched.cpp:765] Scheduler::registered took 27128ns 3: I1201 01:27:47.442750 17337 hierarchical.cpp:293] Added framework d7ac1d4c-38ff-4a0c-9c6c-44a867cdac57-0000 3: I1201 01:27:47.442996 17337 hierarchical.cpp:1431] Performed allocation for 0 agents in 94395ns 3: I1201 01:27:47.446166 17338 state.cpp:64] Recovering state from '/tmp/PartitionTest_PartitionedSlaveReregistrationMasterFailover_bxc0C8/meta' 3: I1201 01:27:47.493692 17336 task_status_update_manager.cpp:207] Recovering task status update manager 3: I1201 01:27:47.494524 17331 containerizer.cpp:672] Recovering containerizer 3: I1201 01:27:47.496757 17329 provisioner.cpp:493] Provisioner recovery complete 3: I1201 01:27:47.497386 17330 slave.cpp:6513] Finished recovery 3: I1201 01:27:47.498594 17330 slave.cpp:999] New master detected at [email protected]:52315 3: I1201 01:27:47.498742 17330 slave.cpp:1034] Detecting new master 3: I1201 01:27:47.498775 17335 task_status_update_manager.cpp:181] Pausing sending task status updates 3: I1201 01:27:47.500123 17339 slave.cpp:1061] Authenticating with master [email protected]:52315 3: I1201 01:27:47.500246 17339 slave.cpp:1070] Using default CRAM-MD5 authenticatee 3: I1201 01:27:47.500687 17339 authenticatee.cpp:121] Creating new client SASL connection 3: I1201 01:27:47.501157 17339 master.cpp:8589] Authenticating slave(110)@172.17.0.4:52315 3: I1201 01:27:47.501379 17339 authenticator.cpp:414] Starting authentication session for crammd5-authenticatee(239)@172.17.0.4:52315 3: I1201 01:27:47.501701 17330 authenticator.cpp:98] Creating new server SASL connection 3: I1201 01:27:47.502135 17330 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5 3: I1201 01:27:47.502162 17330 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5' 3: I1201 01:27:47.502298 17330 authenticator.cpp:204] Received SASL authentication start 3: I1201 01:27:47.502360 17330 authenticator.cpp:326] Authentication requires more steps 3: I1201 01:27:47.502485 17330 authenticatee.cpp:259] Received SASL authentication step 3: I1201 01:27:47.502619 17330 authenticator.cpp:232] Received SASL authentication step 3: I1201 01:27:47.502651 17330 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: '3e44a50127b8' server FQDN: '3e44a50127b8' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false 3: I1201 01:27:47.502671 17330 auxprop.cpp:181] Looking up auxiliary property '*userPassword' 3: I1201 01:27:47.502718 17330 auxprop.cpp:181] Looking up auxiliary property '*cmusaslsecretCRAM-MD5' 3: I1201 01:27:47.502750 17330 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: '3e44a50127b8' server FQDN: '3e44a50127b8' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true 3: I1201 01:27:47.502765 17330 auxprop.cpp:131] Skipping auxiliary property '*userPassword' since SASL_AUXPROP_AUTHZID == true 3: I1201 01:27:47.502773 17330 auxprop.cpp:131] Skipping auxiliary property '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true 3: I1201 01:27:47.502786 17330 authenticator.cpp:318] Authentication success 3: I1201 01:27:47.503032 17327 authenticatee.cpp:299] Authentication success 3: I1201 01:27:47.503155 17327 master.cpp:8619] Successfully authenticated principal 'test-principal' at slave(110)@172.17.0.4:52315 3: I1201 01:27:47.503273 17327 authenticator.cpp:432] Authentication session cleanup for crammd5-authenticatee(239)@172.17.0.4:52315 3: I1201 01:27:47.503710 17324 slave.cpp:1153] Successfully authenticated with master [email protected]:52315 3: I1201 01:27:47.504006 17324 slave.cpp:1696] Will retry registration in 4.592972ms if necessary 3: I1201 01:27:47.504463 17324 master.cpp:6042] Received register agent message from slave(110)@172.17.0.4:52315 (3e44a50127b8) 3: I1201 01:27:47.504616 17324 master.cpp:3878] Authorizing agent with principal 'test-principal' 3: I1201 01:27:47.505347 17339 master.cpp:6104] Authorized registration of agent at slave(110)@172.17.0.4:52315 (3e44a50127b8) 3: I1201 01:27:47.505496 17339 master.cpp:6197] Registering agent at slave(110)@172.17.0.4:52315 (3e44a50127b8) with id d7ac1d4c-38ff-4a0c-9c6c-44a867cdac57-S0 3: I1201 01:27:47.506109 17339 registrar.cpp:495] Applied 1 operations in 87912ns; attempting to update the registry 3: I1201 01:27:47.507171 17327 registrar.cpp:552] Successfully updated the registry in 0ns 3: I1201 01:27:47.507544 17327 master.cpp:6246] Admitted agent d7ac1d4c-38ff-4a0c-9c6c-44a867cdac57-S0 at slave(110)@172.17.0.4:52315 (3e44a50127b8) 3: I1201 01:27:47.508576 17327 master.cpp:6282] Registered agent d7ac1d4c-38ff-4a0c-9c6c-44a867cdac57-S0 at slave(110)@172.17.0.4:52315 (3e44a50127b8) with cpus:2; mem:1024; disk:3701220; ports:[31000-32000] 3: I1201 01:27:47.509222 17335 slave.cpp:1199] Registered with master [email protected]:52315; given agent ID d7ac1d4c-38ff-4a0c-9c6c-44a867cdac57-S0 3: I1201 01:27:47.509371 17328 task_status_update_manager.cpp:188] Resuming sending task status updates 3: I1201 01:27:47.509436 17327 hierarchical.cpp:553] Added agent d7ac1d4c-38ff-4a0c-9c6c-44a867cdac57-S0 (3e44a50127b8) with cpus:2; mem:1024; disk:3701220; ports:[31000-32000] (allocated: {}) 3: I1201 01:27:47.509774 17335 slave.cpp:1219] Checkpointing SlaveInfo to '/tmp/PartitionTest_PartitionedSlaveReregistrationMasterFailover_bxc0C8/meta/slaves/d7ac1d4c-38ff-4a0c-9c6c-44a867cdac57-S0/slave.info' 3: I1201 01:27:47.510339 17335 slave.cpp:1298] Forwarding total oversubscribed resources {} 3: I1201 01:27:47.510799 17335 master.cpp:7049] Received update of agent d7ac1d4c-38ff-4a0c-9c6c-44a867cdac57-S0 at slave(110)@172.17.0.4:52315 (3e44a50127b8) with total oversubscribed resources {} 3: I1201 01:27:47.511121 17335 master.cpp:7092] Ignoring update on agent d7ac1d4c-38ff-4a0c-9c6c-44a867cdac57-S0 at slave(110)@172.17.0.4:52315 (3e44a50127b8) as it reports no changes 3: I1201 01:27:47.511225 17327 hierarchical.cpp:1431] Performed allocation for 1 agents in 1.601066ms 3: I1201 01:27:47.511827 17335 master.cpp:8419] Sending 1 offers to framework d7ac1d4c-38ff-4a0c-9c6c-44a867cdac57-0000 (default) at [email protected]:52315 3: I1201 01:27:47.512451 17335 sched.cpp:921] Scheduler::resourceOffers took 89626ns 3: I1201 01:27:47.515102 17339 master.cpp:10370] Removing offer d7ac1d4c-38ff-4a0c-9c6c-44a867cdac57-O0 3: I1201 01:27:47.515270 17339 master.cpp:4236] Processing ACCEPT call for offers: [ d7ac1d4c-38ff-4a0c-9c6c-44a867cdac57-O0 ] on agent d7ac1d4c-38ff-4a0c-9c6c-44a867cdac57-S0 at slave(110)@172.17.0.4:52315 (3e44a50127b8) for framework d7ac1d4c-38ff-4a0c-9c6c-44a867cdac57-0000 (default) at [email protected]:52315 3: I1201 01:27:47.515386 17339 master.cpp:3605] Authorizing framework principal 'test-principal' to launch task 5f515f79-1f3d-4281-b501-8cc32f180ce0 3: I1201 01:27:47.518388 17339 master.cpp:11130] Adding task 5f515f79-1f3d-4281-b501-8cc32f180ce0 with resources cpus(allocated: *):1; mem(allocated: *):512 on agent d7ac1d4c-38ff-4a0c-9c6c-44a867cdac57-S0 at slave(110)@172.17.0.4:52315 (3e44a50127b8) write /dev/stdout: resource temporarily unavailable + docker rmi mesos-1512087808-25901 Error response from daemon: conflict: unable to remove repository reference "mesos-1512087808-25901" (must force) - container 3e44a50127b8 is using its referenced image 31cede56d1dc Build step 'Execute shell' marked build as failure Not sending mail to unregistered user [email protected] Not sending mail to unregistered user [email protected]
