[ 
https://issues.apache.org/jira/browse/MESOS-6233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15514527#comment-15514527
 ] 

Alex Kaplan commented on MESOS-6233:
------------------------------------

Additional logs:
{code}
Sep 22 20:03:25 <agent_id> mesos-master[x.x78]: I0922 20:03:25.194582 104504 
replica.cpp:673] Replica in EMPTY status received a broadcasted recover request 
from (20)@x.x.9.1:5050
Sep 22 20:03:25 <agent_id> mesos-master[x.x78]: I0922 20:03:25.194675 104520 
recover.cpp:197] Received a recover response from a replica in EMPTY status
Sep 22 20:03:25 <agent_id> mesos-master[x.x78]: I0922 20:03:25.349588 104504 
replica.cpp:673] Replica in EMPTY status received a broadcasted recover request 
from (23)@x.x.9.1:5050
Sep 22 20:03:25 <agent_id> mesos-master[x.x78]: I0922 20:03:25.349642 104512 
recover.cpp:197] Received a recover response from a replica in EMPTY status
Sep 22 20:03:41 <agent_id> mesos-master[x.x78]: I0922 20:03:41.352486 104529 
recover.cpp:110] Unable to finish the recover protocol in 10secs, retrying
Sep 22 20:03:41 <agent_id> mesos-master[x.x78]: I0922 20:03:41.352586 104529 
recover.cpp:110] Unable to finish the recover protocol in 10secs, retrying
Sep 22 20:03:41 <agent_id> mesos-master[x.x78]: I0922 20:03:41.352995 104510 
replica.cpp:673] Replica in EMPTY status received a broadcasted recover request 
from (26)@x.x.9.1:5050
Sep 22 20:03:41 <agent_id> mesos-master[x.x78]: I0922 20:03:41.353251 104525 
recover.cpp:197] Received a recover response from a replica in EMPTY status
Sep 22 20:03:41 <agent_id> mesos-master[x.x78]: I0922 20:03:41.353310 104510 
replica.cpp:673] Replica in EMPTY status received a broadcasted recover request 
from (29)@x.x.9.1:5050
Sep 22 20:03:41 <agent_id> mesos-master[x.x78]: I0922 20:03:41.353536 x.x94 
recover.cpp:197] Received a recover response from a replica in EMPTY status
Sep 22 20:03:57 <agent_id> mesos-master[x.x78]: I0922 20:03:57.914526 104529 
recover.cpp:110] Unable to finish the recover protocol in 10secs, retrying
Sep 22 20:03:57 <agent_id> mesos-master[x.x78]: I0922 20:03:57.915133 104529 
recover.cpp:110] Unable to finish the recover protocol in 10secs, retrying
Sep 22 20:03:57 <agent_id> mesos-master[x.x78]: I0922 20:03:57.915585 104502 
replica.cpp:673] Replica in EMPTY status received a broadcasted recover request 
from (32)@x.x.9.1:5050
Sep 22 20:03:57 <agent_id> mesos-master[x.x78]: I0922 20:03:57.915774 104525 
recover.cpp:197] Received a recover response from a replica in EMPTY status
Sep 22 20:03:57 <agent_id> mesos-master[x.x78]: I0922 20:03:57.915876 104514 
replica.cpp:673] Replica in EMPTY status received a broadcasted recover request 
from (35)@x.x.9.1:5050
Sep 22 20:03:57 <agent_id> mesos-master[x.x78]: I0922 20:03:57.916045 104510 
recover.cpp:197] Received a recover response from a replica in EMPTY status
Sep 22 20:04:21 <agent_id> mesos-master[x.x78]: I0922 20:04:21.397429 104529 
recover.cpp:110] Unable to finish the recover protocol in 10secs, retrying
Sep 22 20:04:21 <agent_id> mesos-master[x.x78]: I0922 20:04:21.397532 104529 
recover.cpp:110] Unable to finish the recover protocol in 10secs, retrying
Sep 22 20:04:21 <agent_id> mesos-master[x.x78]: I0922 20:04:21.397794 104523 
replica.cpp:673] Replica in EMPTY status received a broadcasted recover request 
from (38)@x.x.9.1:5050
Sep 22 20:04:21 <agent_id> mesos-master[x.x78]: I0922 20:04:21.397994 x.x82 
replica.cpp:673] Replica in EMPTY status received a broadcasted recover request 
from (13)@x.x.11.1:5050
Sep 22 20:04:21 <agent_id> mesos-master[x.x78]: I0922 20:04:21.398031 x.x82 
replica.cpp:673] Replica in EMPTY status received a broadcasted recover request 
from (41)@x.x.9.1:5050
Sep 22 20:04:21 <agent_id> mesos-master[x.x78]: I0922 20:04:21.398175 x.x87 
recover.cpp:197] Received a recover response from a replica in EMPTY status
Sep 22 20:04:21 <agent_id> mesos-master[x.x78]: I0922 20:04:21.398236 104516 
recover.cpp:197] Received a recover response from a replica in EMPTY status
Sep 22 20:04:29 <agent_id> mesos-master[x.x78]: I0922 20:04:29.4x.x6 x.x97 
replica.cpp:673] Replica in EMPTY status received a broadcasted recover request 
from (12)@x.x.11.1:5050
Sep 22 20:04:29 <agent_id> mesos-master[x.x78]: I0922 20:04:29.449352 x.x90 
replica.cpp:673] Replica in EMPTY status received a broadcasted recover request 
from (14)@x.x.11.1:5050
Sep 22 20:04:29 <agent_id> mesos-master[x.x78]: I0922 20:04:29.449399 x.x93 
replica.cpp:673] Replica in EMPTY status received a broadcasted recover request 
from (22)@x.x.11.1:5050
Sep 22 20:04:29 <agent_id> mesos-master[x.x78]: I0922 20:04:29.872488 x.x84 
manager.cpp:201] Overlay master overlay-master@x.x.11.1:5050 has exited
Sep 22 20:04:29 <agent_id> mesos-master[x.x78]: W0922 20:04:29.872529 x.x84 
manager.cpp:204] Overlay master disconnected! Waiting for a new overlay master 
to be detected
Sep 22 20:04:29 <agent_id> mesos-master[x.x78]: I0922 20:04:29.872550 x.x84 
manager.cpp:208] Moving overlay-master@x.x.11.1:5050 to `REGISTERING` state.
Sep 22 20:04:37 <agent_id> mesos-master[x.x78]: I0922 20:04:37.898635 104507 
replica.cpp:673] Replica in EMPTY status received a broadcasted recover request 
from (16)@x.x.11.1:5050
Sep 22 20:04:37 <agent_id> mesos-master[x.x78]: I0922 20:04:37.899080 104529 
recover.cpp:110] Unable to finish the recover protocol in 10secs, retrying
Sep 22 20:04:37 <agent_id> mesos-master[x.x78]: I0922 20:04:37.899253 104529 
recover.cpp:110] Unable to finish the recover protocol in 10secs, retrying
Sep 22 20:04:37 <agent_id> mesos-master[x.x78]: I0922 20:04:37.899842 x.x90 
replica.cpp:673] Replica in EMPTY status received a broadcasted recover request 
from (48)@x.x.9.1:5050
Sep 22 20:04:37 <agent_id> mesos-master[x.x78]: I0922 20:04:37.900182 104526 
recover.cpp:197] Received a recover response from a replica in EMPTY status
Sep 22 20:04:37 <agent_id> mesos-master[x.x78]: I0922 20:04:37.900218 x.x84 
replica.cpp:673] Replica in EMPTY status received a broadcasted recover request 
from (50)@x.x.9.1:5050
Sep 22 20:04:37 <agent_id> mesos-master[x.x78]: I0922 20:04:37.900280 x.x98 
recover.cpp:197] Received a recover response from a replica in EMPTY status
Sep 22 20:04:37 <agent_id> mesos-master[x.x78]: E0922 20:04:37.900727 104529 
process.cpp:2104] Failed to shutdown socket with fd 46: Transport endpoint is 
not connected
Sep 22 20:04:40 <agent_id> mesos-master[x.x78]: I0922 20:04:40.013473 104525 
network.hpp:430] ZooKeeper group memberships changed
Sep 22 20:04:40 <agent_id> mesos-master[x.x78]: I0922 20:04:40.013525 104505 
group.cpp:706] Trying to get '/mesos/overlay_log_replicas/0000003289' in 
ZooKeeper
Sep 22 20:04:40 <agent_id> mesos-master[x.x78]: I0922 20:04:40.013557 104520 
detector.cpp:152] Detected a new leader: (id='2760')
Sep 22 20:04:40 <agent_id> mesos-master[x.x78]: I0922 20:04:40.013662 104523 
network.hpp:430] ZooKeeper group memberships changed
Sep 22 20:04:40 <agent_id> mesos-master[x.x78]: I0922 20:04:40.013720 104528 
detector.cpp:152] Detected a new leader: (id='2760')
Sep 22 20:04:40 <agent_id> mesos-master[x.x78]: I0922 20:04:40.013746 104503 
group.cpp:706] Trying to get '/mesos/json.info_0000002760' in ZooKeeper
Sep 22 20:04:40 <agent_id> mesos-master[x.x78]: I0922 20:04:40.013777 x.x95 
group.cpp:706] Trying to get '/mesos/json.info_0000002760' in ZooKeeper
Sep 22 20:04:40 <agent_id> mesos-master[x.x78]: I0922 20:04:40.013813 x.x86 
group.cpp:706] Trying to get '/mesos/log_replicas/0000003284' in ZooKeeper
Sep 22 20:04:40 <agent_id> mesos-master[x.x78]: I0922 20:04:40.014015 104528 
network.hpp:478] ZooKeeper group PIDs: { log-replica(2)@x.x.9.1:5050 }
Sep 22 20:04:40 <agent_id> mesos-master[x.x78]: I0922 20:04:40.014298 x.x97 
network.hpp:478] ZooKeeper group PIDs: { log-replica(1)@x.x.9.1:5050 }
Sep 22 20:04:40 <agent_id> mesos-master[x.x78]: I0922 20:04:40.014318 x.x99 
zookeeper.cpp:259] A new leading master (UPID=master@x.x.9.1:5050) is detected
Sep 22 20:04:40 <agent_id> mesos-master[x.x78]: I0922 20:04:40.014444 104526 
zookeeper.cpp:259] A new leading master (UPID=master@x.x.9.1:5050) is detected
Sep 22 20:04:40 <agent_id> mesos-master[x.x78]: I0922 20:04:40.014487 x.x90 
master.cpp:1847] The newly elected leader is master@x.x.9.1:5050 with id 
184cd1f8-d3ff-4c90-8559-0ac41ea81616
Sep 22 20:04:40 <agent_id> mesos-master[x.x78]: I0922 20:04:40.014503 x.x90 
master.cpp:1860] Elected as the leading master!
Sep 22 20:04:40 <agent_id> mesos-master[x.x78]: I0922 20:04:40.014504 x.x87 
manager.cpp:397] Detected new overlay master at overlay-master@x.x.9.1:5050
Sep 22 20:04:40 <agent_id> mesos-master[x.x78]: I0922 20:04:40.014511 x.x90 
master.cpp:1547] Recovering from registrar
Sep 22 20:04:40 <agent_id> mesos-master[x.x78]: I0922 20:04:40.014555 x.x86 
manager.cpp:782] Got registration from pid: overlay-agent@x.x.9.1:5050
Sep 22 20:04:40 <agent_id> mesos-master[x.x78]: I0922 20:04:40.014582 x.x86 
manager.cpp:787] overlay-master moving to `RECOVERING` state . Hence, not 
sending an update to agentoverlay-agent@x.x.9.1:5050
Sep 22 20:04:40 <agent_id> mesos-master[x.x78]: I0922 20:04:40.014590 104522 
registrar.cpp:332] Recovering registrar
Sep 22 20:04:45 <agent_id> mesos-master[x.x78]: I0922 20:04:45.911886 104525 
logfmt.cpp:164] dstip=x.x.9.1 type=audit timestamp=2016-09-22 
20:04:45.911847936+00:00 reason="Valid authorization token" 
uid="dcos_mesos_dns" object="/master/state.json" authorizer="mesos-master" 
action="GET" result=allow srcip=x.x.11.1 dstport=5050 srcport=51602
Sep 22 20:04:45 <agent_id> mesos-master[x.x78]: I0922 20:04:45.912039 x.x84 
http.cpp:381] HTTP GET for /master/state.json from x.x.11.1:51602 with 
User-Agent='Mesos-DNS'
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.121378 104522 
network.hpp:430] ZooKeeper group memberships changed
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.121448 104523 
group.cpp:706] Trying to get '/mesos/overlay_log_replicas/0000003289' in 
ZooKeeper
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.121789 104523 
group.cpp:706] Trying to get '/mesos/overlay_log_replicas/0000003291' in 
ZooKeeper
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.122149 104502 
network.hpp:478] ZooKeeper group PIDs: { log-replica(1)@x.x.11.1:5050, 
log-replica(1)@x.x.9.1:5050 }
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.309017 104509 
network.hpp:430] ZooKeeper group memberships changed
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.309114 104502 
group.cpp:706] Trying to get '/mesos/log_replicas/0000003284' in ZooKeeper
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.309340 104502 
group.cpp:706] Trying to get '/mesos/log_replicas/0000003286' in ZooKeeper
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.309658 x.x81 
network.hpp:478] ZooKeeper group PIDs: { log-replica(2)@x.x.11.1:5050, 
log-replica(2)@x.x.9.1:5050 }
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.732125 x.x89 
network.hpp:430] ZooKeeper group memberships changed
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.732303 104507 
group.cpp:706] Trying to get '/mesos/overlay_log_replicas/0000003289' in 
ZooKeeper
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.732770 104507 
group.cpp:706] Trying to get '/mesos/overlay_log_replicas/0000003291' in 
ZooKeeper
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.733115 104507 
group.cpp:706] Trying to get '/mesos/overlay_log_replicas/0000003292' in 
ZooKeeper
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.733594 x.x88 
network.hpp:478] ZooKeeper group PIDs: { log-replica(1)@x.x.11.1:5050, 
log-replica(1)@x.x.9.1:5050, log-replica(1)@x.x.11.1:5050 }
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.886611 104503 
network.hpp:430] ZooKeeper group memberships changed
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.886776 x.x99 
group.cpp:706] Trying to get '/mesos/log_replicas/0000003284' in ZooKeeper
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.887078 x.x99 
group.cpp:706] Trying to get '/mesos/log_replicas/0000003286' in ZooKeeper
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.887272 x.x99 
group.cpp:706] Trying to get '/mesos/log_replicas/0000003287' in ZooKeeper
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.887572 104500 
network.hpp:478] ZooKeeper group PIDs: { log-replica(2)@x.x.11.1:5050, 
log-replica(2)@x.x.9.1:5050, log-replica(2)@x.x.11.1:5050 }
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.919270 104529 
recover.cpp:110] Unable to finish the recover protocol in 10secs, retrying
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.919317 104528 
manager.cpp:782] Got registration from pid: overlay-agent@x.x.9.1:5050
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.919333 104529 
recover.cpp:110] Unable to finish the recover protocol in 10secs, retrying
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.919342 104528 
manager.cpp:795] overlay-master in `RECOVERING` state . Hence, not sending an 
update to agentoverlay-agent@x.x.9.1:5050
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.919360 104528 
manager.cpp:782] Got registration from pid: overlay-agent@x.x.9.1:5050
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.919368 104528 
manager.cpp:795] overlay-master in `RECOVERING` state . Hence, not sending an 
update to agentoverlay-agent@x.x.9.1:5050
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.921273 104523 
logfmt.cpp:164] dstip=x.x.9.1 type=audit timestamp=2016-09-22 
20:04:53.921229056+00:00 reason="Valid authorization token" 
uid="dcos_mesos_dns" object="/master/state.json" authorizer="mesos-master" 
action="GET" result=allow srcip=x.x.11.1 dstport=5050 srcport=34076
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.921572 104523 
logfmt.cpp:164] dstip=x.x.9.1 type=audit timestamp=2016-09-22 
20:04:53.921534976+00:00 reason="Valid authorization token" 
uid="dcos_mesos_dns" object="/master/state.json" authorizer="mesos-master" 
action="GET" result=allow srcip=x.x.9.1 dstport=5050 srcport=35372
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.921723 x.x86 
http.cpp:381] HTTP GET for /master/state.json from x.x.9.1:35372 with 
User-Agent='Mesos-DNS'
Sep 22 20:04:53 <agent_id> mesos-master[x.x78]: I0922 20:04:53.921826 x.x86 
http.cpp:381] HTTP GET for /master/state.json from x.x.11.1:34076 with 
User-Agent='Mesos-DNS'
Sep 22 20:05:08 <agent_id> mesos-master[x.x78]: I0922 20:05:08.023877 104522 
network.hpp:430] ZooKeeper group memberships changed
Sep 22 20:05:08 <agent_id> mesos-master[x.x78]: I0922 20:05:08.023988 x.x82 
network.hpp:430] ZooKeeper group memberships changed
Sep 22 20:05:08 <agent_id> mesos-master[x.x78]: I0922 20:05:08.024041 x.x88 
group.cpp:706] Trying to get '/mesos/overlay_log_replicas/0000003289' in 
ZooKeeper
Sep 22 20:05:08 <agent_id> mesos-master[x.x78]: I0922 20:05:08.024093 x.x87 
group.cpp:706] Trying to get '/mesos/log_replicas/0000003284' in ZooKeeper
Sep 22 20:05:08 <agent_id> mesos-master[x.x78]: I0922 20:05:08.024291 x.x88 
group.cpp:706] Trying to get '/mesos/overlay_log_replicas/0000003292' in 
ZooKeeper
Sep 22 20:05:08 <agent_id> mesos-master[x.x78]: I0922 20:05:08.024492 x.x87 
group.cpp:706] Trying to get '/mesos/log_replicas/0000003287' in ZooKeeper
Sep 22 20:05:08 <agent_id> mesos-master[x.x78]: I0922 20:05:08.024902 104511 
network.hpp:478] ZooKeeper group PIDs: { log-replica(1)@x.x.9.1:5050, 
log-replica(1)@x.x.11.1:5050 }
Sep 22 20:05:08 <agent_id> mesos-master[x.x78]: I0922 20:05:08.025151 104506 
network.hpp:478] ZooKeeper group PIDs: { log-replica(2)@x.x.9.1:5050, 
log-replica(2)@x.x.11.1:5050 }
Sep 22 20:05:12 <agent_id> mesos-master[x.x78]: I0922 20:05:12.016031 104522 
network.hpp:430] ZooKeeper group memberships changed
Sep 22 20:05:12 <agent_id> mesos-master[x.x78]: I0922 20:05:12.016280 104527 
network.hpp:430] ZooKeeper group memberships changed
Sep 22 20:05:12 <agent_id> mesos-master[x.x78]: I0922 20:05:12.016337 x.x98 
group.cpp:706] Trying to get '/mesos/log_replicas/0000003284' in ZooKeeper
Sep 22 20:05:12 <agent_id> mesos-master[x.x78]: I0922 20:05:12.016345 x.x82 
group.cpp:706] Trying to get '/mesos/overlay_log_replicas/0000003289' in 
ZooKeeper
Sep 22 20:05:12 <agent_id> mesos-master[x.x78]: I0922 20:05:12.016741 104522 
network.hpp:478] ZooKeeper group PIDs: { log-replica(1)@x.x.9.1:5050 }
Sep 22 20:05:12 <agent_id> mesos-master[x.x78]: I0922 20:05:12.016783 104510 
network.hpp:478] ZooKeeper group PIDs: { log-replica(2)@x.x.9.1:5050 }
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]: I0922 20:05:17.947398 104529 
recover.cpp:110] Unable to finish the recover protocol in 10secs, retrying
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]: I0922 20:05:17.947479 104529 
recover.cpp:110] Unable to finish the recover protocol in 10secs, retrying
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]: I0922 20:05:17.947873 104511 
manager.cpp:782] Got registration from pid: overlay-agent@x.x.9.1:5050
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]: I0922 20:05:17.947899 104511 
manager.cpp:795] overlay-master in `RECOVERING` state . Hence, not sending an 
update to agentoverlay-agent@x.x.9.1:5050
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]: I0922 20:05:17.947917 104511 
manager.cpp:782] Got registration from pid: overlay-agent@x.x.9.1:5050
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]: I0922 20:05:17.947926 104511 
manager.cpp:795] overlay-master in `RECOVERING` state . Hence, not sending an 
update to agentoverlay-agent@x.x.9.1:5050
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]: I0922 20:05:17.947978 x.x95 
manager.cpp:782] Got registration from pid: overlay-agent@x.x.4.1:5051
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]: I0922 20:05:17.948004 x.x95 
manager.cpp:795] overlay-master in `RECOVERING` state . Hence, not sending an 
update to agentoverlay-agent@x.x.4.1:5051
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]: F0922 20:05:17.948120 104529 
process.cpp:2243] Check failed: sockets.count(from_fd) > 0
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]: *** Check failure stack trace: 
***
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]:     @     0x7fc1908829fd  
google::LogMessage::Fail()
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]:     @     0x7fc19088482d  
google::LogMessage::SendToLog()
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]:     @     0x7fc1908825ec  
google::LogMessage::Flush()
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]:     @     0x7fc190885129  
google::LogMessageFatal::~LogMessageFatal()
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]:     @     0x7fc1908171dd  
process::SocketManager::swap_implementing_socket()
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]:     @     0x7fc19081aa90  
process::SocketManager::link_connect()
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]:     @     0x7fc1908227f9  
_ZNSt17_Function_handlerIFvRKN7process6FutureI7NothingEEEZNKS3_5onAnyISt5_BindIFSt7_Mem_fnIMNS0_13SocketManagerEFvS5_NS0_7network6SocketERKNS0_4UPIDEEEPSA_St12_PlaceholderILi1EESC_SD_EEvEES5_OT_NS3_6PreferEEUlS5_E_E9_M_invokeERKSt9_Any_dataS5_
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]:     @           0x41eb26  
_ZN7process8internal3runISt8functionIFvRKNS_6FutureI7NothingEEEEJRS5_EEEvRKSt6vectorIT_SaISC_EEDpOT0_
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]:     @           0x42a36f  
process::Future<>::fail()
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]:     @     0x7fc19085283c  
process::network::LibeventSSLSocketImpl::event_callback()
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]:     @     0x7fc190852f17  
process::network::LibeventSSLSocketImpl::event_callback()
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]:     @     0x7fc18d616631  
bufferevent_run_deferred_callbacks_locked
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]:     @     0x7fc18d60cc5d  
event_base_loop
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]:     @     0x7fc190865a1d  
process::EventLoop::run()
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]:     @     0x7fc18eeabd73  
(unknown)
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]:     @     0x7fc18e6a852c  
(unknown)
Sep 22 20:05:17 <agent_id> mesos-master[x.x78]:     @     0x7fc18e3e61dd  
(unknown)
Sep 22 20:05:18 <agent_id> systemd[1]: [0;1;39mdcos-mesos-master.service: Main 
process exited, code=killed, status=6/ABRT[0m
Sep 22 20:05:18 <agent_id> systemd[1]: [0;1;39mdcos-mesos-master.service: Unit 
entered failed state.[0m
Sep 22 20:05:18 <agent_id> systemd[1]: [0;1;39mdcos-mesos-master.service: 
Failed with result 'signal'.[0m
Sep 22 20:05:33 <agent_id> systemd[1]: dcos-mesos-master.service: Service 
hold-off time over, scheduling restart.
Sep 22 20:05:33 <agent_id> systemd[1]: Stopped Mesos Master: DC/OS Mesos Master 
Service.
Sep 22 20:05:33 <agent_id> systemd[1]: Starting Mesos Master: DC/OS Mesos 
Master Service...
Sep 22 20:05:33 <agent_id> ping[104808]: PING ready.spartan (127.0.0.1) 56(84) 
bytes of data.
Sep 22 20:05:33 <agent_id> ping[104808]: 64 bytes from <agent_id> (127.0.0.1): 
icmp_seq=1 ttl=64 time=0.025 ms
Sep 22 20:05:33 <agent_id> ping[104808]: --- ready.spartan ping statistics ---
Sep 22 20:05:33 <agent_id> ping[104808]: 1 packets transmitted, 1 received, 0% 
packet loss, time 0ms
Sep 22 20:05:33 <agent_id> ping[104808]: rtt min/avg/max/mdev = 
0.025/0.025/0.025/0.000 ms
Sep 22 20:05:33 <agent_id> bash[104810]: /proc/sys/net/ipv4/conf/all/rp_filter: 
2
Sep 22 20:05:33 <agent_id> bash[104810]: 
/proc/sys/net/ipv4/conf/cbr0/rp_filter: 2
Sep 22 20:05:33 <agent_id> bash[104810]: 
/proc/sys/net/ipv4/conf/default/rp_filter: 2
Sep 22 20:05:33 <agent_id> bash[104810]: 
/proc/sys/net/ipv4/conf/dummy0/rp_filter: 2
Sep 22 20:05:33 <agent_id> bash[104810]: 
/proc/sys/net/ipv4/conf/eno1/rp_filter: 2
Sep 22 20:05:33 <agent_id> bash[104810]: 
/proc/sys/net/ipv4/conf/eno2/rp_filter: 2
Sep 22 20:05:33 <agent_id> bash[104810]: 
/proc/sys/net/ipv4/conf/eno3/rp_filter: 2
Sep 22 20:05:33 <agent_id> bash[104810]: 
/proc/sys/net/ipv4/conf/eno4/rp_filter: 2
Sep 22 20:05:33 <agent_id> bash[104810]: /proc/sys/net/ipv4/conf/lo/rp_filter: 2
Sep 22 20:05:33 <agent_id> bash[104810]: 
/proc/sys/net/ipv4/conf/spartan/rp_filter: 2
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Clearing proxy environment 
variables
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] PID 3794 has command line 
[b'/opt/mesosphere/active/java/usr/java/bin/java', 
b'-Dzookeeper.log.dir=/var/lib/dcos/exhibitor/zookeeper', 
b'-Dzookeeper.root.logger=INFO,CONSOLE', b'-cp', 
b'/opt/mesosphere/active/exhibitor/usr/zookeeper/bin/../build/classes:/opt/mesosphere/active/exhibitor/usr/zookeeper/bin/../build/lib/*.jar:/opt/mesosphere/active/exhibitor/usr/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/opt/mesosphere/active/exhibitor/usr/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/opt/mesosphere/active/exhibitor/usr/zookeeper/bin/../lib/netty-3.7.0.Final.jar:/opt/mesosphere/active/exhibitor/usr/zookeeper/bin/../lib/log4j-systemd-journal-appender-1.3.2.jar:/opt/mesosphere/active/exhibitor/usr/zookeeper/bin/../lib/log4j-1.2.16.jar:/opt/mesosphere/active/exhibitor/usr/zookeeper/bin/../lib/jna-4.2.2.jar:/opt/mesosphere/active/exhibitor/usr/zookeeper/bin/../lib/jline-0.9.94.jar:/opt/mesosphere/active/exhibitor/usr/zookeeper/bin/../zookeeper-3.4.8.jar:/opt/mesosphere/active/exhibitor/usr/zookeeper/bin/../src/java/lib/*.jar:/var/lib/dcos/exhibitor/conf:',
 
b'-Dzookeeper.DigestAuthenticationProvider.superDigest=super:lK75jTNcA+U9vtVEw5vB51mj/w4=',
 b'-Dcom.sun.management.jmxremote', 
b'-Dcom.sun.management.jmxremote.local.only=false', 
b'org.apache.zookeeper.server.quorum.QuorumPeerMain', 
b'/var/lib/dcos/exhibitor/conf/zoo.cfg']
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] PID file hasn't been 
modified. ZK still seems to be at that PID.
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Shortcut succeeeded, 
assuming local zk is in good config state, not waiting for quorum.
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Preparing directory 
/run/dcos
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Preparing directory 
/run/dcos/etc
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Preparing directory 
/run/dcos/etc/3dt
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Preparing directory 
/run/dcos/etc/marathon
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Preparing directory 
/run/dcos/etc/mesos
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Preparing directory 
/run/dcos/etc/mesos-dns
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Preparing directory 
/run/dcos/etc/dcos-ca
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Preparing directory 
/run/dcos/etc/metronome
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Preparing directory 
/run/dcos/etc/history-service
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Preparing directory 
/run/dcos/etc/signal-service
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Preparing directory 
/run/dcos/pki/tls/private
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Preparing directory 
/run/dcos/pki/tls/certs
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Preparing directory 
/run/dcos/pki/CA/certs
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Preparing directory 
/run/dcos/pki/CA/private
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Using super credentials 
for Zookeeper
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Connecting to 
127.0.0.1:2181
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Zookeeper connection 
established, state: CONNECTED
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [DEBUG] bootstrapping 
dcos-mesos-master
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Initializing ACLs for 
znode /
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] ensure_zk_path(/, 
[ACL(perms=5, acl_list=['READ', 'CREATE'], id=Id(scheme='world', id='anyone')), 
ACL(perms=31, acl_list=['ALL'], id=Id(scheme='ip', id='127.0.0.1'))])
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Initializing ACLs for 
znode /cosmos
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] ensure_zk_path(/cosmos, 
[ACL(perms=31, acl_list=['ALL'], id=Id(scheme='world', id='anyone'))])
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Initializing ACLs for 
znode /dcos
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] ensure_zk_path(/dcos, 
[ACL(perms=1, acl_list=['READ'], id=Id(scheme='world', id='anyone'))])
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Initializing ACLs for 
znode /dcos/vault
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/vault, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='world', id='anyone'))])
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Initializing ACLs for 
znode /zookeeper
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] ensure_zk_path(/zookeeper, 
[ACL(perms=1, acl_list=['READ'], id=Id(scheme='world', id='anyone'))])
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Initializing ACLs for 
znode /zookeeper/quotas
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/zookeeper/quotas, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='world', id='anyone'))])
Sep 22 20:05:33 <agent_id> bootstrap[104823]: [INFO] Creating master secrets 
with user dcos-master
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/master/secrets, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-master:fMkMgKtR6Fl+wYKfdJg75Th6Vsc='))])
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/master/secrets/private_keys
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/master/secrets/private_keys already exists
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/master/secrets/zk, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-master:fMkMgKtR6Fl+wYKfdJg75Th6Vsc='))])
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/master/secrets/zk/dcos_bouncer
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/master/secrets/zk/dcos_bouncer already exists
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/master/secrets/zk, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-master:fMkMgKtR6Fl+wYKfdJg75Th6Vsc='))])
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/master/secrets/zk/dcos_mesos_master
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/master/secrets/zk/dcos_mesos_master already exists
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/master/secrets/zk, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-master:fMkMgKtR6Fl+wYKfdJg75Th6Vsc='))])
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/master/secrets/zk/dcos_secrets
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/master/secrets/zk/dcos_secrets already exists
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/master/secrets/zk, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-master:fMkMgKtR6Fl+wYKfdJg75Th6Vsc='))])
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/master/secrets/zk/dcos_cosmos
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/master/secrets/zk/dcos_cosmos already exists
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/master/secrets/zk, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-master:fMkMgKtR6Fl+wYKfdJg75Th6Vsc='))])
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/master/secrets/zk/dcos_ca
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/master/secrets/zk/dcos_ca already exists
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/master/secrets/zk, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-master:fMkMgKtR6Fl+wYKfdJg75Th6Vsc='))])
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/master/secrets/zk/dcos_marathon
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/master/secrets/zk/dcos_marathon already exists
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/master/secrets/zk, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-master:fMkMgKtR6Fl+wYKfdJg75Th6Vsc='))])
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/master/secrets/zk/dcos_vault_default
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/master/secrets/zk/dcos_vault_default already exists
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/master/secrets/zk, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-master:fMkMgKtR6Fl+wYKfdJg75Th6Vsc='))])
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/master/secrets/zk/dcos_metronome
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/master/secrets/zk/dcos_metronome already exists
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/master/secrets/CA, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-master:fMkMgKtR6Fl+wYKfdJg75Th6Vsc='))])
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/master/secrets/CA/RootCA
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/master/secrets/CA/RootCA already exists
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/master/secrets/services, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-master:fMkMgKtR6Fl+wYKfdJg75Th6Vsc='))])
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/master/secrets/services/dcos_spartan_master
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/master/secrets/services/dcos_spartan_master already exists
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/master/secrets/services, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-master:fMkMgKtR6Fl+wYKfdJg75Th6Vsc='))])
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/master/secrets/services/dcos_navstar_master
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/master/secrets/services/dcos_navstar_master already exists
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/master/secrets/services, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-master:fMkMgKtR6Fl+wYKfdJg75Th6Vsc='))])
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/master/secrets/services/dcos_history_service
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/master/secrets/services/dcos_history_service already exists
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/master/secrets/services, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-master:fMkMgKtR6Fl+wYKfdJg75Th6Vsc='))])
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/master/secrets/services/dcos_3dt_master
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/master/secrets/services/dcos_3dt_master already exists
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/master/secrets/services, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-master:fMkMgKtR6Fl+wYKfdJg75Th6Vsc='))])
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/master/secrets/services/dcos_marathon
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/master/secrets/services/dcos_marathon already exists
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/master/secrets/services, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-master:fMkMgKtR6Fl+wYKfdJg75Th6Vsc='))])
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/master/secrets/services/dcos_networking_api_master
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/master/secrets/services/dcos_networking_api_master already exists
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/master/secrets/services, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-master:fMkMgKtR6Fl+wYKfdJg75Th6Vsc='))])
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/master/secrets/services/dcos_minuteman_master
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/master/secrets/services/dcos_minuteman_master already exists
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/master/secrets/services, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-master:fMkMgKtR6Fl+wYKfdJg75Th6Vsc='))])
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/master/secrets/services/dcos_mesos_dns
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/master/secrets/services/dcos_mesos_dns already exists
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/master/secrets/services, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-master:fMkMgKtR6Fl+wYKfdJg75Th6Vsc='))])
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/master/secrets/services/dcos_signal_service
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/master/secrets/services/dcos_signal_service already exists
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/master/secrets/services, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-master:fMkMgKtR6Fl+wYKfdJg75Th6Vsc='))])
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/master/secrets/services/dcos_adminrouter
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/master/secrets/services/dcos_adminrouter already exists
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/master/secrets/services, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-master:fMkMgKtR6Fl+wYKfdJg75Th6Vsc='))])
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/master/secrets/services/dcos_metronome
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/master/secrets/services/dcos_metronome already exists
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] ensure_zk_path(/mesos, 
[ACL(perms=1, acl_list=['READ'], id=Id(scheme='world', id='anyone')), 
ACL(perms=31, acl_list=['ALL'], id=Id(scheme='ip', id='127.0.0.1')), 
ACL(perms=31, acl_list=['ALL'], id=Id(scheme='digest', 
id='dcos_mesos_master:IeBqBqLqnGlC3FoMAKYACqaZlIQ='))])
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Writing Mesos Master ZK 
credentials to /run/dcos/etc/mesos-master
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Opening /run/dcos/etc
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Opened /run/dcos/etc with 
fd 8
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Locking fd 8
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Locked fd 8
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Writing 
/run/dcos/etc/mesos-master with mode 600
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Unlocked fd 8
Sep 22 20:05:35 <agent_id> bootstrap[104823]: [INFO] Closing /run/dcos/etc with 
fd 8
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/agent/secrets/services, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-agent:ZT4/4Ug5cTT0jcU4PP/2D9W6zOs='))])
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/agent/secrets/services/dcos_navstar_agent
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/agent/secrets/services/dcos_navstar_agent already exists
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/agent/secrets/services, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-agent:ZT4/4Ug5cTT0jcU4PP/2D9W6zOs='))])
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/agent/secrets/services/dcos_spartan_agent
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/agent/secrets/services/dcos_spartan_agent already exists
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/agent/secrets/services, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-agent:ZT4/4Ug5cTT0jcU4PP/2D9W6zOs='))])
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/agent/secrets/services/dcos_mesos_agent_public
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/agent/secrets/services/dcos_mesos_agent_public already exists
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/agent/secrets/services, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-agent:ZT4/4Ug5cTT0jcU4PP/2D9W6zOs='))])
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/agent/secrets/services/dcos_minuteman_agent
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/agent/secrets/services/dcos_minuteman_agent already exists
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/agent/secrets/services, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-agent:ZT4/4Ug5cTT0jcU4PP/2D9W6zOs='))])
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/agent/secrets/services/dcos_mesos_agent
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/agent/secrets/services/dcos_mesos_agent already exists
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/agent/secrets/services, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-agent:ZT4/4Ug5cTT0jcU4PP/2D9W6zOs='))])
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/agent/secrets/services/dcos_agent
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/agent/secrets/services/dcos_agent already exists
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/agent/secrets/services, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-agent:ZT4/4Ug5cTT0jcU4PP/2D9W6zOs='))])
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/agent/secrets/services/dcos_adminrouter_agent
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/agent/secrets/services/dcos_adminrouter_agent already exists
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] 
ensure_zk_path(/dcos/agent/secrets/services, [ACL(perms=1, acl_list=['READ'], 
id=Id(scheme='digest', id='dcos-agent:ZT4/4Ug5cTT0jcU4PP/2D9W6zOs='))])
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] Reaching consensus about 
znode /dcos/agent/secrets/services/dcos_3dt_agent
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] Consensus znode 
/dcos/agent/secrets/services/dcos_3dt_agent already exists
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] Creating service account 
dcos_agent
Sep 22 20:05:36 <agent_id> bootstrap[104823]: [INFO] Starting new HTTP 
connection (1): 127.0.0.1
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] status_code=409
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Service account dcos_agent 
already exists
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Starting new HTTP 
connection (1): 127.0.0.1
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Creating service account 
dcos_mesos_agent
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Starting new HTTP 
connection (1): 127.0.0.1
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] status_code=409
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Service account 
dcos_mesos_agent already exists
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Starting new HTTP 
connection (1): 127.0.0.1
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Creating service account 
dcos_mesos_agent_public
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Starting new HTTP 
connection (1): 127.0.0.1
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] status_code=409
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Service account 
dcos_mesos_agent_public already exists
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Starting new HTTP 
connection (1): 127.0.0.1
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Creating service account 
dcos_adminrouter_agent
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Starting new HTTP 
connection (1): 127.0.0.1
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] status_code=409
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Service account 
dcos_adminrouter_agent already exists
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Starting new HTTP 
connection (1): 127.0.0.1
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Creating service account 
dcos_3dt_agent
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Starting new HTTP 
connection (1): 127.0.0.1
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] status_code=409
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Service account 
dcos_3dt_agent already exists
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Starting new HTTP 
connection (1): 127.0.0.1
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Creating service account 
dcos_minuteman_agent
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Starting new HTTP 
connection (1): 127.0.0.1
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] status_code=409
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Service account 
dcos_minuteman_agent already exists
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Starting new HTTP 
connection (1): 127.0.0.1
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Creating service account 
dcos_navstar_agent
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Starting new HTTP 
connection (1): 127.0.0.1
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] status_code=409
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Service account 
dcos_navstar_agent already exists
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Starting new HTTP 
connection (1): 127.0.0.1
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Creating service account 
dcos_spartan_agent
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Starting new HTTP 
connection (1): 127.0.0.1
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] status_code=409
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Service account 
dcos_spartan_agent already exists
Sep 22 20:05:37 <agent_id> bootstrap[104823]: [INFO] Starting new HTTP 
connection (1): 127.0.0.1
Sep 22 20:05:37 <agent_id> systemd[1]: Started Mesos Master: DC/OS Mesos Master 
Service.
Sep 22 20:05:37 <agent_id> mesos-master[104837]: WARNING: Logging before 
InitGoogleLogging() is written to STDERR
Sep 22 20:05:37 <agent_id> mesos-master[104837]: I0922 20:05:37.972128 104837 
main.cpp:263] Build: 2016-09-13 x.x2:04 by
Sep 22 20:05:37 <agent_id> mesos-master[104837]: I0922 20:05:37.972210 104837 
main.cpp:264] Version: 1.0.1
Sep 22 20:05:37 <agent_id> mesos-master[104837]: I0922 20:05:37.972216 104837 
main.cpp:271] Git SHA: a22b5dd552168a8dd925682f3b349059e10fc3d3
Sep 22 20:05:37 <agent_id> mesos-master[104837]: W0922 20:05:37.973675 104837 
openssl.cpp:399] Failed SSL connections will be downgraded to a non-SSL socket
Sep 22 20:05:37 <agent_id> mesos-master[104837]: W0922 20:05:37.973877 104837 
process.cpp:880] Failed SSL connections will be downgraded to a non-SSL socket
Sep 22 20:05:37 <agent_id> mesos-master[104837]: I0922 20:05:37.975872 104837 
logging.cpp:194] INFO level logging started!
Sep 22 20:05:37 <agent_id> mesos-master[104837]: W0922 20:05:37.976052 104837 
main.cpp:290] Loaded deprecated flag 'max_slave_ping_timeouts'
Sep 22 20:05:37 <agent_id> mesos-master[104837]: W0922 20:05:37.976161 104837 
main.cpp:290] Loaded deprecated flag 'slave_removal_rate_limit'
Sep 22 20:05:37 <agent_id> mesos-master[104837]: I0922 20:05:37.992913 104837 
manager.cpp:1422] Overlay master parameter 
'master_config=/opt/mesosphere/etc/overlay/config/master.json'
Sep 22 20:05:37 <agent_id> mesos-master[104837]: I0922 20:05:37.993281 104837 
manager.cpp:606] Configuring overlay network:dcos
Sep 22 20:05:37 <agent_id> mesos-master[104837]: I0922 20:05:37.993336 104837 
manager.cpp:650] Initializing the replicated log.
Sep 22 20:05:37 <agent_id> mesos-master[104837]: I0922 20:05:37.993355 104837 
manager.cpp:686] Using replicated log with zookeeper URL 
zk://dcos_mesos_master:6EIjIPV6aX3U8M8kTpSkqXw4e5bZBGZizVV02rdBQdjOv9TzvH4BimMFmmVLe8Kv@127.0.0.1:2181/mesos
 with a quorum of 2 for leader election.
Sep 22 20:05:38 <agent_id> mesos-master[104837]: I0922 20:05:38.094033 104837 
leveldb.cpp:174] Opened db in 100.622609ms
Sep 22 20:05:38 <agent_id> mesos-master[104837]: I0922 20:05:38.x.x81 104837 
leveldb.cpp:181] Compacted db in 14.315233ms
Sep 22 20:05:38 <agent_id> mesos-master[104837]: I0922 20:05:38.x.x58 104837 
leveldb.cpp:196] Created db iterator in 6554ns
Sep 22 20:05:38 <agent_id> mesos-master[104837]: I0922 20:05:38.x.x67 104837 
leveldb.cpp:202] Seeked to beginning of db in 672ns
Sep 22 20:05:38 <agent_id> mesos-master[104837]: I0922 20:05:38.x.x72 104837 
leveldb.cpp:271] Iterated through 0 keys in the db in 288ns
Sep 22 20:05:38 <agent_id> mesos-master[104837]: I0922 20:05:38.x.x97 104837 
replica.cpp:779] Replica recovered with log positions 0 -> 0 with 1 holes and 0 
unlearned
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,108:104837(0x7f7a49d4d700):ZOO_INFO@log_env@726: Client 
environment:zookeeper.version=zookeeper C client 3.4.8
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,108:104837(0x7f7a49d4d700):ZOO_INFO@log_env@730: Client 
environment:host.name=<agent_id>
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,108:104837(0x7f7a49d4d700):ZOO_INFO@log_env@737: Client 
environment:os.name=Linux
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,108:104837(0x7f7a49d4d700):ZOO_INFO@log_env@738: Client 
environment:os.arch=4.6.3-coreos
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,108:104837(0x7f7a49d4d700):ZOO_INFO@log_env@739: Client 
environment:os.version=#2 SMP Tue Aug 23 02:14:16 UTC 2016
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,108:104837(0x7f7a4ed57700):ZOO_INFO@log_env@726: Client 
environment:zookeeper.version=zookeeper C client 3.4.8
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,108:104837(0x7f7a4ed57700):ZOO_INFO@log_env@730: Client 
environment:host.name=<agent_id>
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,108:104837(0x7f7a4ed57700):ZOO_INFO@log_env@737: Client 
environment:os.name=Linux
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,108:104837(0x7f7a4ed57700):ZOO_INFO@log_env@738: Client 
environment:os.arch=4.6.3-coreos
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,108:104837(0x7f7a4ed57700):ZOO_INFO@log_env@739: Client 
environment:os.version=#2 SMP Tue Aug 23 02:14:16 UTC 2016
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,109:104837(0x7f7a4ed57700):ZOO_INFO@log_env@747: Client 
environment:user.name=(null)
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,109:104837(0x7f7a49d4d700):ZOO_INFO@log_env@747: Client 
environment:user.name=(null)
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,109:104837(0x7f7a4ed57700):ZOO_INFO@log_env@755: Client 
environment:user.home=/root
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,109:104837(0x7f7a4ed57700):ZOO_INFO@log_env@767: Client 
environment:user.dir=/
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,109:104837(0x7f7a4ed57700):ZOO_INFO@zookeeper_init@800: Initiating 
client connection, host=127.0.0.1:2181 sessionTimeout=10000 
watcher=0x7f7a5e765d00 sessionId=0 sessionPasswd=<null> context=0x7f7a38000ae0 
flags=0
Sep 22 20:05:38 <agent_id> mesos-master[104837]: I0922 20:05:38.x.x31 104887 
log.cpp:107] Attempting to join replica to ZooKeeper group
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,109:104837(0x7f7a49d4d700):ZOO_INFO@log_env@755: Client 
environment:user.home=/root
Sep 22 20:05:38 <agent_id> mesos-master[104837]: I0922 20:05:38.x.x78 104837 
manager.cpp:871] Overlay agent parameter 
'agent_config=/opt/mesosphere/etc/overlay/config/agent-master.json'
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,109:104837(0x7f7a49d4d700):ZOO_INFO@log_env@767: Client 
environment:user.dir=/
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,109:104837(0x7f7a49d4d700):ZOO_INFO@zookeeper_init@800: Initiating 
client connection, host=127.0.0.1:2181 sessionTimeout=10000 
watcher=0x7f7a5e765d00 sessionId=0 sessionPasswd=<null> context=0x7f79f8000930 
flags=0
Sep 22 20:05:38 <agent_id> mesos-master[104837]: I0922 20:05:38.x.x73 104876 
manager.cpp:759] Adding route for 'overlay-master/state'
Sep 22 20:05:38 <agent_id> mesos-master[104837]: I0922 20:05:38.110025 104870 
recover.cpp:451] Starting replica recovery
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,110:104837(0x7f7a47d49700):ZOO_INFO@log_env@726: Client 
environment:zookeeper.version=zookeeper C client 3.4.8
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,110:104837(0x7f7a47d49700):ZOO_INFO@log_env@730: Client 
environment:host.name=<agent_id>
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,110:104837(0x7f7a47d49700):ZOO_INFO@log_env@737: Client 
environment:os.name=Linux
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,110:104837(0x7f7a47d49700):ZOO_INFO@log_env@738: Client 
environment:os.arch=4.6.3-coreos
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,110:104837(0x7f7a47d49700):ZOO_INFO@log_env@739: Client 
environment:os.version=#2 SMP Tue Aug 23 02:14:16 UTC 2016
Sep 22 20:05:38 <agent_id> mesos-master[104837]: I0922 20:05:38.110203 104870 
recover.cpp:477] Replica is in EMPTY status
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,110:104837(0x7f7a265f2700):ZOO_INFO@check_events@1728: initiated 
connection to server [127.0.0.1:2181]
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,110:104837(0x7f7a275f4700):ZOO_INFO@check_events@1728: initiated 
connection to server [127.0.0.1:2181]
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,110:104837(0x7f7a47d49700):ZOO_INFO@log_env@747: Client 
environment:user.name=(null)
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,111:104837(0x7f7a47d49700):ZOO_INFO@log_env@755: Client 
environment:user.home=/root
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,111:104837(0x7f7a47d49700):ZOO_INFO@log_env@767: Client 
environment:user.dir=/
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,111:104837(0x7f7a47d49700):ZOO_INFO@zookeeper_init@800: Initiating 
client connection, host=127.0.0.1:2181 sessionTimeout=10000 
watcher=0x7f7a5e765d00 sessionId=0 sessionPasswd=<null> context=0x7f79e8001f30 
flags=0
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,111:104837(0x7f7a255f0700):ZOO_INFO@check_events@1728: initiated 
connection to server [127.0.0.1:2181]
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,123:104837(0x7f7a265f2700):ZOO_INFO@check_events@1775: session 
establishment complete on server [127.0.0.1:2181], sessionId=0x3574e2d05adb704, 
negotiated timeout=10000
Sep 22 20:05:38 <agent_id> mesos-master[104837]: I0922 20:05:38.124027 104867 
group.cpp:349] Group process (group(1)@x.x.9.1:5050) connected to ZooKeeper
Sep 22 20:05:38 <agent_id> mesos-master[104837]: I0922 20:05:38.124076 104867 
group.cpp:837] Syncing group operations: queue size (joins, cancels, datas) = 
(0, 0, 0)
Sep 22 20:05:38 <agent_id> mesos-master[104837]: I0922 20:05:38.124088 104867 
group.cpp:403] Authenticating with ZooKeeper using digest
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,129:104837(0x7f7a275f4700):ZOO_INFO@check_events@1775: session 
establishment complete on server [127.0.0.1:2181], sessionId=0x3574e2d05adb705, 
negotiated timeout=10000
Sep 22 20:05:38 <agent_id> mesos-master[104837]: I0922 20:05:38.129184 104871 
group.cpp:349] Group process (group(2)@x.x.9.1:5050) connected to ZooKeeper
Sep 22 20:05:38 <agent_id> mesos-master[104837]: I0922 20:05:38.129216 104871 
group.cpp:837] Syncing group operations: queue size (joins, cancels, datas) = 
(1, 0, 0)
Sep 22 20:05:38 <agent_id> mesos-master[104837]: I0922 20:05:38.129225 104871 
group.cpp:403] Authenticating with ZooKeeper using digest
Sep 22 20:05:38 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:38,129:104837(0x7f7a255f0700):ZOO_INFO@check_events@1775: session 
establishment complete on server [127.0.0.1:2181], sessionId=0x3574e2d05adb706, 
negotiated timeout=10000
Sep 22 20:05:38 <agent_id> mesos-master[104837]: I0922 20:05:38.129473 104865 
group.cpp:349] Group process (group(3)@x.x.9.1:5050) connected to ZooKeeper
Sep 22 20:05:38 <agent_id> mesos-master[104837]: I0922 20:05:38.129508 104865 
group.cpp:837] Syncing group operations: queue size (joins, cancels, datas) = 
(0, 0, 0)
Sep 22 20:05:38 <agent_id> mesos-master[104837]: I0922 20:05:38.129521 104865 
group.cpp:403] Authenticating with ZooKeeper using digest
Sep 22 20:05:41 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:41,446:104837(0x7f7a265f2700):ZOO_INFO@auth_completion_func@1300: 
Authentication scheme digest succeeded
Sep 22 20:05:41 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:41,446:104837(0x7f7a275f4700):ZOO_INFO@auth_completion_func@1300: 
Authentication scheme digest succeeded
Sep 22 20:05:41 <agent_id> mesos-master[104837]: I0922 20:05:41.446926 104867 
group.cpp:427] Trying to create path '/mesos/overlay_log_replicas' in ZooKeeper
Sep 22 20:05:41 <agent_id> mesos-master[104837]: I0922 20:05:41.447043 104871 
group.cpp:427] Trying to create path '/mesos/overlay_log_replicas' in ZooKeeper
Sep 22 20:05:41 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:41,449:104837(0x7f7a255f0700):ZOO_INFO@auth_completion_func@1300: 
Authentication scheme digest succeeded
Sep 22 20:05:41 <agent_id> mesos-master[104837]: I0922 20:05:41.449318 104865 
group.cpp:427] Trying to create path '/mesos' in ZooKeeper
Sep 22 20:05:41 <agent_id> mesos-master[104837]: I0922 20:05:41.454278 104860 
network.hpp:430] ZooKeeper group memberships changed
Sep 22 20:05:41 <agent_id> mesos-master[104837]: I0922 20:05:41.454349 104854 
group.cpp:706] Trying to get '/mesos/overlay_log_replicas/0000003293' in 
ZooKeeper
Sep 22 20:05:41 <agent_id> mesos-master[104837]: I0922 20:05:41.454555 104854 
group.cpp:706] Trying to get '/mesos/overlay_log_replicas/0000003294' in 
ZooKeeper
Sep 22 20:05:41 <agent_id> mesos-master[104837]: I0922 20:05:41.454640 104875 
detector.cpp:152] Detected a new leader: (id='2763')
Sep 22 20:05:41 <agent_id> mesos-master[104837]: I0922 20:05:41.463635 104858 
group.cpp:706] Trying to get '/mesos/json.info_0000002763' in ZooKeeper
Sep 22 20:05:41 <agent_id> mesos-master[104837]: I0922 20:05:41.464010 104854 
group.cpp:706] Trying to get '/mesos/overlay_log_replicas/0000003295' in 
ZooKeeper
Sep 22 20:05:41 <agent_id> mesos-master[104837]: I0922 20:05:41.464095 104871 
zookeeper.cpp:259] A new leading master (UPID=master@x.x.11.1:5050) is detected
Sep 22 20:05:41 <agent_id> mesos-master[104837]: I0922 20:05:41.464279 104868 
network.hpp:478] ZooKeeper group PIDs: { log-replica(1)@x.x.11.1:5050, 
log-replica(1)@x.x.9.1:5050, log-replica(1)@x.x.11.1:5050 }
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.001029 104851 
manager.cpp:180] Initializing overlay agent manager
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.001037 104837 
manager.cpp:850] Overlay agent is ready
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.001206 104837 
main.cpp:370] Using 'HierarchicalDRF' allocator
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.001278 104847 
manager.cpp:397] Detected new overlay master at overlay-master@x.x.11.1:5050
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.042414 104837 
leveldb.cpp:174] Opened db in 41.142916ms
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.054388 104837 
leveldb.cpp:181] Compacted db in 11.932746ms
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.054451 104837 
leveldb.cpp:196] Created db iterator in 3474ns
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.054461 104837 
leveldb.cpp:202] Seeked to beginning of db in 687ns
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.054466 104837 
leveldb.cpp:271] Iterated through 0 keys in the db in 251ns
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.054479 104837 
replica.cpp:779] Replica recovered with log positions 0 -> 0 with 1 holes and 0 
unlearned
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,054:104837(0x7f7a53d61700):ZOO_INFO@log_env@726: Client 
environment:zookeeper.version=zookeeper C client 3.4.8
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,054:104837(0x7f7a53d61700):ZOO_INFO@log_env@730: Client 
environment:host.name=<agent_id>
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,054:104837(0x7f7a53d61700):ZOO_INFO@log_env@737: Client 
environment:os.name=Linux
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,054:104837(0x7f7a53d61700):ZOO_INFO@log_env@738: Client 
environment:os.arch=4.6.3-coreos
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.054653 104867 
log.cpp:107] Attempting to join replica to ZooKeeper group
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,054:104837(0x7f7a4253e700):ZOO_INFO@log_env@726: Client 
environment:zookeeper.version=zookeeper C client 3.4.8
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,054:104837(0x7f7a53d61700):ZOO_INFO@log_env@739: Client 
environment:os.version=#2 SMP Tue Aug 23 02:14:16 UTC 2016
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,054:104837(0x7f7a4253e700):ZOO_INFO@log_env@730: Client 
environment:host.name=<agent_id>
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,054:104837(0x7f7a4253e700):ZOO_INFO@log_env@737: Client 
environment:os.name=Linux
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,054:104837(0x7f7a4253e700):ZOO_INFO@log_env@738: Client 
environment:os.arch=4.6.3-coreos
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,054:104837(0x7f7a4253e700):ZOO_INFO@log_env@739: Client 
environment:os.version=#2 SMP Tue Aug 23 02:14:16 UTC 2016
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.054760 104868 
recover.cpp:451] Starting replica recovery
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.054831 104868 
recover.cpp:477] Replica is in EMPTY status
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,054:104837(0x7f7a53d61700):ZOO_INFO@log_env@747: Client 
environment:user.name=(null)
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,054:104837(0x7f7a53d61700):ZOO_INFO@log_env@755: Client 
environment:user.home=/root
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,054:104837(0x7f7a53d61700):ZOO_INFO@log_env@767: Client 
environment:user.dir=/
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,054:104837(0x7f7a53d61700):ZOO_INFO@zookeeper_init@800: Initiating 
client connection, host=127.0.0.1:2181 sessionTimeout=10000 
watcher=0x7f7a5e765d00 sessionId=0 sessionPasswd=<null> context=0x7f7964000950 
flags=0
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,054:104837(0x7f7a4c552700):ZOO_INFO@log_env@726: Client 
environment:zookeeper.version=zookeeper C client 3.4.8
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,054:104837(0x7f7a4c552700):ZOO_INFO@log_env@730: Client 
environment:host.name=<agent_id>
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,054:104837(0x7f7a4c552700):ZOO_INFO@log_env@737: Client 
environment:os.name=Linux
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,054:104837(0x7f7a4c552700):ZOO_INFO@log_env@738: Client 
environment:os.arch=4.6.3-coreos
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,054:104837(0x7f7a4c552700):ZOO_INFO@log_env@739: Client 
environment:os.version=#2 SMP Tue Aug 23 02:14:16 UTC 2016
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,055:104837(0x7f7a59d6d700):ZOO_INFO@log_env@726: Client 
environment:zookeeper.version=zookeeper C client 3.4.8
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,055:104837(0x7f7a59d6d700):ZOO_INFO@log_env@730: Client 
environment:host.name=<agent_id>
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,055:104837(0x7f7a59d6d700):ZOO_INFO@log_env@737: Client 
environment:os.name=Linux
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,055:104837(0x7f7a59d6d700):ZOO_INFO@log_env@738: Client 
environment:os.arch=4.6.3-coreos
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,055:104837(0x7f7a4c552700):ZOO_INFO@log_env@747: Client 
environment:user.name=(null)
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,055:104837(0x7f7a59d6d700):ZOO_INFO@log_env@739: Client 
environment:os.version=#2 SMP Tue Aug 23 02:14:16 UTC 2016
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.055543 104837 
main.cpp:543] Starting Mesos master
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,055:104837(0x7f7a4253e700):ZOO_INFO@log_env@747: Client 
environment:user.name=(null)
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,055:104837(0x7f7a4c552700):ZOO_INFO@log_env@755: Client 
environment:user.home=/root
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,055:104837(0x7f7a4c552700):ZOO_INFO@log_env@767: Client 
environment:user.dir=/
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,055:104837(0x7f7a4c552700):ZOO_INFO@zookeeper_init@800: Initiating 
client connection, host=127.0.0.1:2181 sessionTimeout=10000 
watcher=0x7f7a5e765d00 sessionId=0 sessionPasswd=<null> context=0x7f7a28000f50 
flags=0
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,055:104837(0x7f7a4253e700):ZOO_INFO@log_env@755: Client 
environment:user.home=/root
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,055:104837(0x7f7a0ffff700):ZOO_INFO@check_events@1728: initiated 
connection to server [127.0.0.1:2181]
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,055:104837(0x7f7a4253e700):ZOO_INFO@log_env@767: Client 
environment:user.dir=/
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,055:104837(0x7f7a4253e700):ZOO_INFO@zookeeper_init@800: Initiating 
client connection, host=127.0.0.1:2181 sessionTimeout=10000 
watcher=0x7f7a5e765d00 sessionId=0 sessionPasswd=<null> context=0x7f7980001030 
flags=0
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,055:104837(0x7f7a59d6d700):ZOO_INFO@log_env@747: Client 
environment:user.name=(null)
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,055:104837(0x7f7a59d6d700):ZOO_INFO@log_env@755: Client 
environment:user.home=/root
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,055:104837(0x7f7a59d6d700):ZOO_INFO@log_env@767: Client 
environment:user.dir=/
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,055:104837(0x7f7a59d6d700):ZOO_INFO@zookeeper_init@800: Initiating 
client connection, host=127.0.0.1:2181 sessionTimeout=10000 
watcher=0x7f7a5e765d00 sessionId=0 sessionPasswd=<null> context=0x7f79e0002170 
flags=0
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,056:104837(0x7f7a0effd700):ZOO_INFO@check_events@1728: initiated 
connection to server [127.0.0.1:2181]
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,056:104837(0x7f7a0dffb700):ZOO_INFO@check_events@1728: initiated 
connection to server [127.0.0.1:2181]
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,056:104837(0x7f7a0cff9700):ZOO_INFO@check_events@1728: initiated 
connection to server [127.0.0.1:2181]
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.057487 104869 
master.cpp:375] Master 4be01ec7-9b37-43be-ba3e-d8f70db0a3fe (x.x.9.1) started 
on x.x.9.1:5050
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.057510 104869 
master.cpp:377] Flags at startup: --agent_ping_timeout="15secs" 
--slave_removal_rate_limit="1/20mins" --agent_reregister_timeout="10mins" 
--allocation_interval="1secs" --allocator="HierarchicalDRF" 
--authenticate_agents="false" --authenticate_frameworks="false" 
--authenticate_http_frameworks="false" --authenticate_http_readonly="true" 
--authenticate_http_readwrite="true" 
--authenticators="com_mesosphere_dcos_ClassicRPCAuthenticator" 
--authorizers="local" --cluster="walthamDCOS1.8" --framework_sorter="drf" 
--help="false" --hostname_lookup="false" 
--http_authenticators="com_mesosphere_dcos_http_Authenticator" 
--initialize_driver_logging="true" 
--ip_discovery_command="/opt/mesosphere/bin/detect_ip" 
--log_auto_initialize="true" --log_dir="/var/lib/dcos/mesos/log" 
--logbufsecs="0" --logging_level="INFO" --max_slave_ping_timeouts="20" 
--max_completed_frameworks="50" --max_completed_tasks_per_framework="1000" 
--modules_dir="/opt/mesosphere/etc/mesos-master-modules" 
--offer_timeout="2mins" --port="5050" --quiet="false" --quorum="2" 
--recovery_agent_removal_limit="100%" --registry="replicated_log" 
--registry_fetch_timeout="1mins" --registry_store_timeout="1mins" 
--registry_strict="false" --root_submissions="true" --user_sorter="drf" 
--version="false" 
--webui_dir="/opt/mesosphere/packages/mesos--2b6cac8d498de56ee4c2eb9a89900b5ca06e270b/share/mesos/webui"
 --weights="" --work_dir="/var/lib/dcos/mesos/master" 
--zk="zk://dcos_mesos_master:6EIjIPV6aX3U8M8kTpSkqXw4e5bZBGZizVV02rdBQdjOv9TzvH4BimMFmmVLe8Kv@127.0.0.1:2181/mesos"
 --zk_session_timeout="10secs"
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.057643 104869 
master.cpp:429] Master allowing unauthenticated frameworks to register
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.057651 104869 
master.cpp:443] Master allowing unauthenticated agents to register
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.057657 104869 
master.cpp:457] Master allowing HTTP frameworks to register without 
authentication
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,060:104837(0x7f7a0ffff700):ZOO_INFO@check_events@1775: session 
establishment complete on server [127.0.0.1:2181], sessionId=0x3574e2d05adb710, 
negotiated timeout=10000
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.060330 104852 
group.cpp:349] Group process (group(4)@x.x.9.1:5050) connected to ZooKeeper
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.060361 104852 
group.cpp:837] Syncing group operations: queue size (joins, cancels, datas) = 
(0, 0, 0)
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.060374 104852 
group.cpp:403] Authenticating with ZooKeeper using digest
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,065:104837(0x7f7a0effd700):ZOO_INFO@check_events@1775: session 
establishment complete on server [127.0.0.1:2181], sessionId=0x3574e2d05adb711, 
negotiated timeout=10000
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.066030 104856 
group.cpp:349] Group process (group(6)@x.x.9.1:5050) connected to ZooKeeper
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.066056 104856 
group.cpp:837] Syncing group operations: queue size (joins, cancels, datas) = 
(0, 0, 0)
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.066063 104856 
group.cpp:403] Authenticating with ZooKeeper using digest
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,066:104837(0x7f7a0dffb700):ZOO_INFO@check_events@1775: session 
establishment complete on server [127.0.0.1:2181], sessionId=0x3574e2d05adb712, 
negotiated timeout=10000
Sep 22 20:05:46 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:46,066:104837(0x7f7a0cff9700):ZOO_INFO@check_events@1775: session 
establishment complete on server [127.0.0.1:2181], sessionId=0x3574e2d05adb713, 
negotiated timeout=10000
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.066226 104883 
group.cpp:349] Group process (group(5)@x.x.9.1:5050) connected to ZooKeeper
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.066254 104883 
group.cpp:837] Syncing group operations: queue size (joins, cancels, datas) = 
(1, 0, 0)
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.066264 104883 
group.cpp:403] Authenticating with ZooKeeper using digest
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.066596 104843 
group.cpp:349] Group process (group(7)@x.x.9.1:5050) connected to ZooKeeper
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.066627 104843 
group.cpp:837] Syncing group operations: queue size (joins, cancels, datas) = 
(0, 0, 0)
Sep 22 20:05:46 <agent_id> mesos-master[104837]: I0922 20:05:46.066637 104843 
group.cpp:403] Authenticating with ZooKeeper using digest
Sep 22 20:05:49 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:49,392:104837(0x7f7a0ffff700):ZOO_INFO@auth_completion_func@1300: 
Authentication scheme digest succeeded
Sep 22 20:05:49 <agent_id> mesos-master[104837]: I0922 20:05:49.392606 104852 
group.cpp:427] Trying to create path '/mesos/log_replicas' in ZooKeeper
Sep 22 20:05:49 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:49,392:104837(0x7f7a0effd700):ZOO_INFO@auth_completion_func@1300: 
Authentication scheme digest succeeded
Sep 22 20:05:49 <agent_id> mesos-master[104837]: I0922 20:05:49.392740 104856 
group.cpp:427] Trying to create path '/mesos' in ZooKeeper
Sep 22 20:05:49 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:49,393:104837(0x7f7a0dffb700):ZOO_INFO@auth_completion_func@1300: 
Authentication scheme digest succeeded
Sep 22 20:05:49 <agent_id> mesos-master[104837]: I0922 20:05:49.393430 104883 
group.cpp:427] Trying to create path '/mesos/log_replicas' in ZooKeeper
Sep 22 20:05:49 <agent_id> mesos-master[104837]: 2016-09-22 
20:05:49,393:104837(0x7f7a0cff9700):ZOO_INFO@auth_completion_func@1300: 
Authentication scheme digest succeeded
Sep 22 20:05:49 <agent_id> mesos-master[104837]: I0922 20:05:49.393985 104881 
network.hpp:430] ZooKeeper group memberships changed
Sep 22 20:05:49 <agent_id> mesos-master[104837]: I0922 20:05:49.394183 104887 
group.cpp:706] Trying to get '/mesos/log_replicas/0000003288' in ZooKeeper
Sep 22 20:05:49 <agent_id> mesos-master[104837]: I0922 20:05:49.394270 104843 
group.cpp:427] Trying to create path '/mesos' in ZooKeeper
Sep 22 20:05:49 <agent_id> mesos-master[104837]: I0922 20:05:49.403239 104887 
group.cpp:706] Trying to get '/mesos/log_replicas/0000003289' in ZooKeeper
Sep 22 20:05:49 <agent_id> mesos-master[104837]: I0922 20:05:49.403499 104859 
detector.cpp:152] Detected a new leader: (id='2763')
Sep 22 20:05:49 <agent_id> mesos-master[104837]: I0922 20:05:49.403614 104849 
group.cpp:706] Trying to get '/mesos/json.info_0000002763' in ZooKeeper
Sep 22 20:05:49 <agent_id> mesos-master[104837]: I0922 20:05:49.403832 104866 
network.hpp:478] ZooKeeper group PIDs: { log-replica(2)@x.x.11.1:5050, 
log-replica(2)@x.x.11.1:5050 }
Sep 22 20:05:49 <agent_id> mesos-master[104837]: I0922 20:05:49.403944 104842 
zookeeper.cpp:259] A new leading master (UPID=master@x.x.11.1:5050) is detected
Sep 22 20:05:49 <agent_id> mesos-master[104837]: I0922 20:05:49.404078 104875 
network.hpp:430] ZooKeeper group memberships changed
Sep 22 20:05:49 <agent_id> mesos-master[104837]: I0922 20:05:49.404206 104884 
group.cpp:706] Trying to get '/mesos/log_replicas/0000003288' in ZooKeeper
Sep 22 20:05:49 <agent_id> mesos-master[104837]: I0922 20:05:49.404521 104884 
group.cpp:706] Trying to get '/mesos/log_replicas/0000003289' in ZooKeeper
Sep 22 20:05:49 <agent_id> mesos-master[104837]: I0922 20:05:49.404736 104884 
group.cpp:706] Trying to get '/mesos/log_replicas/0000003290' in ZooKeeper
Sep 22 20:05:49 <agent_id> mesos-master[104837]: I0922 20:05:49.404980 104884 
network.hpp:478] ZooKeeper group PIDs: { log-replica(2)@x.x.11.1:5050, 
log-replica(2)@x.x.9.1:5050, log-replica(2)@x.x.11.1:5050 }
Sep 22 20:05:57 <agent_id> mesos-master[104837]: I0922 20:05:57.483232 104867 
replica.cpp:673] Replica in EMPTY status received a broadcasted recover request 
from (13)@x.x.9.1:5050
Sep 22 20:05:57 <agent_id> mesos-master[104837]: I0922 20:05:57.483387 104850 
recover.cpp:197] Received a recover response from a replica in EMPTY status
{code}

> Unable to start Mesos Master
> ----------------------------
>
>                 Key: MESOS-6233
>                 URL: https://issues.apache.org/jira/browse/MESOS-6233
>             Project: Mesos
>          Issue Type: Bug
>          Components: general, master
>    Affects Versions: 1.0.1
>            Reporter: Alex Kaplan
>            Priority: Blocker
>              Labels: mesosphere
>             Fix For: 1.0.2
>
>
> Mesos Version: 1.0.1
> OS: CoreOS 1068
> {code}
> Sep 22 20:05:17 node-44a84215535c mesos-master[104478]: I0922 20:05:17.948004 
> 104495 manager.cpp:795] overlay-master in `RECOVERING` state . Hence, not 
> sending an update to agentoverlay-agent@10.4.4.1:5051
> Sep 22 20:05:17 node-44a84215535c mesos-master[104478]: F0922 20:05:17.948120 
> 104529 process.cpp:2243] Check failed: sockets.count(from_fd) > 0
> Sep 22 20:05:17 node-44a84215535c mesos-master[104478]: *** Check failure 
> stack trace: ***
> Sep 22 20:05:17 node-44a84215535c mesos-master[104478]:     @     
> 0x7fc1908829fd  google::LogMessage::Fail()
> Sep 22 20:05:17 node-44a84215535c mesos-master[104478]:     @     
> 0x7fc19088482d  google::LogMessage::SendToLog()
> Sep 22 20:05:17 node-44a84215535c mesos-master[104478]:     @     
> 0x7fc1908825ec  google::LogMessage::Flush()
> Sep 22 20:05:17 node-44a84215535c mesos-master[104478]:     @     
> 0x7fc190885129  google::LogMessageFatal::~LogMessageFatal()
> Sep 22 20:05:17 node-44a84215535c mesos-master[104478]:     @     
> 0x7fc1908171dd  process::SocketManager::swap_implementing_socket()
> Sep 22 20:05:17 node-44a84215535c mesos-master[104478]:     @     
> 0x7fc19081aa90  process::SocketManager::link_connect()
> Sep 22 20:05:17 node-44a84215535c mesos-master[104478]:     @     
> 0x7fc1908227f9  
> _ZNSt17_Function_handlerIFvRKN7process6FutureI7NothingEEEZNKS3_5onAnyISt5_BindIFSt7_Mem_fnIMNS0_13SocketManagerEFvS5_NS0_7network6SocketERKNS0_4UPIDEEEPSA_St12_PlaceholderILi1EESC_SD_EEvEES5_OT_NS3_6PreferEEUlS5_E_E9_M_invokeERKSt9_Any_dataS5_
> Sep 22 20:05:17 node-44a84215535c mesos-master[104478]:     @           
> 0x41eb26  
> _ZN7process8internal3runISt8functionIFvRKNS_6FutureI7NothingEEEEJRS5_EEEvRKSt6vectorIT_SaISC_EEDpOT0_
> Sep 22 20:05:17 node-44a84215535c mesos-master[104478]:     @           
> 0x42a36f  process::Future<>::fail()
> Sep 22 20:05:17 node-44a84215535c mesos-master[104478]:     @     
> 0x7fc19085283c  process::network::LibeventSSLSocketImpl::event_callback()
> Sep 22 20:05:17 node-44a84215535c mesos-master[104478]:     @     
> 0x7fc190852f17  process::network::LibeventSSLSocketImpl::event_callback()
> Sep 22 20:05:17 node-44a84215535c mesos-master[104478]:     @     
> 0x7fc18d616631  bufferevent_run_deferred_callbacks_locked
> Sep 22 20:05:17 node-44a84215535c mesos-master[104478]:     @     
> 0x7fc18d60cc5d  event_base_loop
> Sep 22 20:05:17 node-44a84215535c mesos-master[104478]:     @     
> 0x7fc190865a1d  process::EventLoop::run()
> Sep 22 20:05:17 node-44a84215535c mesos-master[104478]:     @     
> 0x7fc18eeabd73  (unknown)
> Sep 22 20:05:17 node-44a84215535c mesos-master[104478]:     @     
> 0x7fc18e6a852c  (unknown)
> Sep 22 20:05:17 node-44a84215535c mesos-master[104478]:     @     
> 0x7fc18e3e61dd  (unknown)
> Sep 22 20:05:18 node-44a84215535c systemd[1]: 
> [0;1;39mdcos-mesos-master.service: Main process exited, code=killed, 
> status=6/ABRT
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to