[
https://issues.apache.org/jira/browse/MESOS-1802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997592#comment-14997592
]
Vinod Kone commented on MESOS-1802:
-----------------------------------
This is still happening.
https://builds.apache.org/job/Mesos/COMPILER=gcc,CONFIGURATION=--verbose,OS=ubuntu%3A14.04,label_exp=docker%7C%7CHadoop/1204/changes
{code}
[ RUN ] HealthCheckTest.HealthStatusChange
I1109 19:00:52.094830 29603 leveldb.cpp:176] Opened db in 282.154657ms
I1109 19:00:52.187214 29603 leveldb.cpp:183] Compacted db in 92.28671ms
I1109 19:00:52.187328 29603 leveldb.cpp:198] Created db iterator in 31318ns
I1109 19:00:52.187345 29603 leveldb.cpp:204] Seeked to beginning of db in 3925ns
I1109 19:00:52.187357 29603 leveldb.cpp:273] Iterated through 0 keys in the db
in 386ns
I1109 19:00:52.187417 29603 replica.cpp:780] Replica recovered with log
positions 0 -> 0 with 1 holes and 0 unlearned
I1109 19:00:52.188252 29628 recover.cpp:449] Starting replica recovery
I1109 19:00:52.188861 29628 recover.cpp:475] Replica is in EMPTY status
I1109 19:00:52.190914 29626 replica.cpp:676] Replica in EMPTY status received a
broadcasted recover request from (1864)@172.17.14.49:60889
I1109 19:00:52.191453 29632 recover.cpp:195] Received a recover response from a
replica in EMPTY status
I1109 19:00:52.192078 29632 recover.cpp:566] Updating replica status to STARTING
I1109 19:00:52.199542 29624 master.cpp:367] Master
34506b61-7fc1-4e5e-821c-e5f5650782e0 (61f7c652bdf2) started on
172.17.14.49:60889
I1109 19:00:52.200220 29624 master.cpp:369] Flags at startup: --acls=""
--allocation_interval="1secs" --allocator="HierarchicalDRF"
--authenticate="true" --authenticate_slaves="true" --authenticators="crammd5"
--authorizers="local" --credentials="/tmp/z7oqX0/credentials"
--framework_sorter="drf" --help="false" --hostname_lookup="true"
--initialize_driver_logging="true" --log_auto_initialize="true"
--logbufsecs="0" --logging_level="INFO" --max_slave_ping_timeouts="5"
--quiet="false" --recovery_slave_removal_limit="100%"
--registry="replicated_log" --registry_fetch_timeout="1mins"
--registry_store_timeout="25secs" --registry_strict="true"
--root_submissions="true" --slave_ping_timeout="15secs"
--slave_reregister_timeout="10mins" --user_sorter="drf" --version="false"
--webui_dir="/mesos/mesos-0.26.0/_inst/share/mesos/webui"
--work_dir="/tmp/z7oqX0/master" --zk_session_timeout="10secs"
I1109 19:00:52.200584 29624 master.cpp:414] Master only allowing authenticated
frameworks to register
I1109 19:00:52.200597 29624 master.cpp:419] Master only allowing authenticated
slaves to register
I1109 19:00:52.200606 29624 credentials.hpp:37] Loading credentials for
authentication from '/tmp/z7oqX0/credentials'
I1109 19:00:52.200917 29624 master.cpp:458] Using default 'crammd5'
authenticator
I1109 19:00:52.201328 29624 master.cpp:495] Authorization enabled
I1109 19:00:52.201648 29622 hierarchical.cpp:140] Initialized hierarchical
allocator process
I1109 19:00:52.201715 29622 whitelist_watcher.cpp:79] No whitelist given
I1109 19:00:52.204258 29624 master.cpp:1606] The newly elected leader is
[email protected]:60889 with id 34506b61-7fc1-4e5e-821c-e5f5650782e0
I1109 19:00:52.204290 29624 master.cpp:1619] Elected as the leading master!
I1109 19:00:52.204308 29624 master.cpp:1379] Recovering from registrar
I1109 19:00:52.204457 29637 registrar.cpp:309] Recovering registrar
I1109 19:00:52.257841 29632 leveldb.cpp:306] Persisting metadata (8 bytes) to
leveldb took 65.359249ms
I1109 19:00:52.257933 29632 replica.cpp:323] Persisted replica status to
STARTING
I1109 19:00:52.258283 29631 recover.cpp:475] Replica is in STARTING status
I1109 19:00:52.259827 29630 replica.cpp:676] Replica in STARTING status
received a broadcasted recover request from (1865)@172.17.14.49:60889
I1109 19:00:52.260082 29628 recover.cpp:195] Received a recover response from a
replica in STARTING status
I1109 19:00:52.260671 29630 recover.cpp:566] Updating replica status to VOTING
I1109 19:00:52.369948 29628 leveldb.cpp:306] Persisting metadata (8 bytes) to
leveldb took 109.102187ms
I1109 19:00:52.370046 29628 replica.cpp:323] Persisted replica status to VOTING
I1109 19:00:52.370317 29628 recover.cpp:580] Successfully joined the Paxos group
I1109 19:00:52.370551 29628 recover.cpp:464] Recover process terminated
I1109 19:00:52.371186 29629 log.cpp:661] Attempting to start the writer
I1109 19:00:52.372530 29629 replica.cpp:496] Replica received implicit promise
request from (1866)@172.17.14.49:60889 with proposal 1
I1109 19:00:52.505920 29629 leveldb.cpp:306] Persisting metadata (8 bytes) to
leveldb took 133.338727ms
I1109 19:00:52.505997 29629 replica.cpp:345] Persisted promised to 1
I1109 19:00:52.506805 29634 coordinator.cpp:240] Coordinator attempting to fill
missing positions
I1109 19:00:52.508029 29622 replica.cpp:391] Replica received explicit promise
request from (1867)@172.17.14.49:60889 for position 0 with proposal 2
I1109 19:00:52.587518 29622 leveldb.cpp:343] Persisting action (8 bytes) to
leveldb took 79.418401ms
I1109 19:00:52.587616 29622 replica.cpp:715] Persisted action at 0
I1109 19:00:52.589311 29622 replica.cpp:540] Replica received write request for
position 0 from (1868)@172.17.14.49:60889
I1109 19:00:52.589390 29622 leveldb.cpp:438] Reading position from leveldb took
41754ns
I1109 19:00:52.660668 29622 leveldb.cpp:343] Persisting action (14 bytes) to
leveldb took 71.221465ms
I1109 19:00:52.660756 29622 replica.cpp:715] Persisted action at 0
I1109 19:00:52.661685 29625 replica.cpp:694] Replica received learned notice
for position 0 from @0.0.0.0:0
I1109 19:00:52.706508 29625 leveldb.cpp:343] Persisting action (16 bytes) to
leveldb took 44.76317ms
I1109 19:00:52.706595 29625 replica.cpp:715] Persisted action at 0
I1109 19:00:52.706634 29625 replica.cpp:700] Replica learned NOP action at
position 0
I1109 19:00:52.707479 29625 log.cpp:677] Writer started with ending position 0
I1109 19:00:52.708673 29631 leveldb.cpp:438] Reading position from leveldb took
62378ns
I1109 19:00:52.709760 29628 registrar.cpp:342] Successfully fetched the
registry (0B) in 505.250304ms
I1109 19:00:52.709918 29628 registrar.cpp:441] Applied 1 operations in 46159ns;
attempting to update the 'registry'
I1109 19:00:52.710719 29626 log.cpp:685] Attempting to append 176 bytes to the
log
I1109 19:00:52.710878 29622 coordinator.cpp:350] Coordinator attempting to
write APPEND action at position 1
I1109 19:00:52.711644 29634 replica.cpp:540] Replica received write request for
position 1 from (1869)@172.17.14.49:60889
I1109 19:00:52.757084 29634 leveldb.cpp:343] Persisting action (195 bytes) to
leveldb took 45.376201ms
I1109 19:00:52.757189 29634 replica.cpp:715] Persisted action at 1
I1109 19:00:52.758299 29636 replica.cpp:694] Replica received learned notice
for position 1 from @0.0.0.0:0
I1109 19:00:52.799000 29636 leveldb.cpp:343] Persisting action (197 bytes) to
leveldb took 40.641232ms
I1109 19:00:52.799090 29636 replica.cpp:715] Persisted action at 1
I1109 19:00:52.799129 29636 replica.cpp:700] Replica learned APPEND action at
position 1
I1109 19:00:52.801550 29632 registrar.cpp:486] Successfully updated the
'registry' in 91.542016ms
I1109 19:00:52.801815 29632 registrar.cpp:372] Successfully recovered registrar
I1109 19:00:52.802433 29634 master.cpp:1416] Recovered 0 slaves from the
Registry (137B) ; allowing 10mins for slaves to re-register
I1109 19:00:52.802548 29632 log.cpp:704] Attempting to truncate the log to 1
I1109 19:00:52.802938 29632 coordinator.cpp:350] Coordinator attempting to
write TRUNCATE action at position 2
I1109 19:00:52.804299 29632 replica.cpp:540] Replica received write request for
position 2 from (1870)@172.17.14.49:60889
I1109 19:00:52.832676 29632 leveldb.cpp:343] Persisting action (16 bytes) to
leveldb took 28.304266ms
I1109 19:00:52.832762 29632 replica.cpp:715] Persisted action at 2
I1109 19:00:52.833796 29627 replica.cpp:694] Replica received learned notice
for position 2 from @0.0.0.0:0
I1109 19:00:52.881970 29627 leveldb.cpp:343] Persisting action (18 bytes) to
leveldb took 48.117789ms
I1109 19:00:52.882125 29627 leveldb.cpp:401] Deleting ~1 keys from leveldb took
71118ns
I1109 19:00:52.882160 29627 replica.cpp:715] Persisted action at 2
I1109 19:00:52.882472 29627 replica.cpp:700] Replica learned TRUNCATE action at
position 2
I1109 19:00:52.886776 29603 containerizer.cpp:142] Using isolation:
posix/cpu,posix/mem,filesystem/posix
W1109 19:00:52.887220 29603 backend.cpp:50] Failed to create 'bind' backend:
BindBackend requires root privileges
I1109 19:00:52.892428 29636 slave.cpp:191] Slave started on
59)@172.17.14.49:60889
I1109 19:00:52.892465 29636 slave.cpp:192] Flags at startup:
--appc_store_dir="/tmp/mesos/store/appc" --authenticatee="crammd5"
--cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false"
--cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false"
--cgroups_root="mesos" --container_disk_watch_interval="15secs"
--containerizers="mesos"
--credential="/tmp/HealthCheckTest_HealthStatusChange_xdqiRg/credential"
--default_role="*" --disk_watch_interval="1mins" --docker="docker"
--docker_kill_orphans="true"
--docker_local_archives_dir="/tmp/mesos/images/docker" --docker_puller="local"
--docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock"
--docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker"
--enforce_container_disk_quota="false" --executor_registration_timeout="1mins"
--executor_shutdown_grace_period="5secs"
--fetcher_cache_dir="/tmp/HealthCheckTest_HealthStatusChange_xdqiRg/fetch"
--fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks"
--gc_disk_headroom="0.1" --hadoop_home="" --help="false"
--hostname_lookup="true" --image_provisioner_backend="copy"
--initialize_driver_logging="true" --isolation="posix/cpu,posix/mem"
--launcher_dir="/mesos/mesos-0.26.0/_build/src" --logbufsecs="0"
--logging_level="INFO" --oversubscribed_resources_interval="15secs"
--perf_duration="10secs" --perf_interval="1mins"
--qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect"
--recovery_timeout="15mins" --registration_backoff_factor="10ms"
--resources="cpus:2;mem:1024;disk:1024;ports:[31000-32000]"
--revocable_cpu_low_priority="true" --sandbox_directory="/mnt/mesos/sandbox"
--strict="true" --switch_user="true"
--systemd_runtime_directory="/run/systemd/system" --version="false"
--work_dir="/tmp/HealthCheckTest_HealthStatusChange_xdqiRg"
I1109 19:00:52.892906 29636 credentials.hpp:85] Loading credential for
authentication from '/tmp/HealthCheckTest_HealthStatusChange_xdqiRg/credential'
I1109 19:00:52.893141 29636 slave.cpp:322] Slave using credential for:
test-principal
I1109 19:00:52.893776 29636 slave.cpp:392] Slave resources: cpus(*):2;
mem(*):1024; disk(*):1024; ports(*):[31000-32000]
I1109 19:00:52.893852 29636 slave.cpp:400] Slave attributes: [ ]
I1109 19:00:52.893865 29636 slave.cpp:405] Slave hostname: 61f7c652bdf2
I1109 19:00:52.893874 29636 slave.cpp:410] Slave checkpoint: true
I1109 19:00:52.894933 29603 sched.cpp:166] Version: 0.26.0
I1109 19:00:52.895546 29629 state.cpp:54] Recovering state from
'/tmp/HealthCheckTest_HealthStatusChange_xdqiRg/meta'
I1109 19:00:52.895649 29630 sched.cpp:264] New master detected at
[email protected]:60889
I1109 19:00:52.895730 29630 sched.cpp:320] Authenticating with master
[email protected]:60889
I1109 19:00:52.895752 29630 sched.cpp:327] Using default CRAM-MD5 authenticatee
I1109 19:00:52.896075 29630 authenticatee.cpp:123] Creating new client SASL
connection
I1109 19:00:52.896335 29626 status_update_manager.cpp:202] Recovering status
update manager
I1109 19:00:52.896419 29636 master.cpp:5150] Authenticating
[email protected]:60889
I1109 19:00:52.896545 29623 authenticator.cpp:415] Starting authentication
session for crammd5_authenticatee(164)@172.17.14.49:60889
I1109 19:00:52.896879 29623 authenticator.cpp:100] Creating new server SASL
connection
I1109 19:00:52.897099 29623 authenticatee.cpp:214] Received SASL authentication
mechanisms: CRAM-MD5
I1109 19:00:52.897128 29623 authenticatee.cpp:240] Attempting to authenticate
with mechanism 'CRAM-MD5'
I1109 19:00:52.897286 29637 authenticator.cpp:205] Received SASL authentication
start
I1109 19:00:52.897351 29637 authenticator.cpp:327] Authentication requires more
steps
I1109 19:00:52.897400 29630 containerizer.cpp:384] Recovering containerizer
I1109 19:00:52.897434 29637 authenticatee.cpp:260] Received SASL authentication
step
I1109 19:00:52.897555 29637 authenticator.cpp:233] Received SASL authentication
step
I1109 19:00:52.897585 29637 auxprop.cpp:109] Request to lookup properties for
user: 'test-principal' realm: '61f7c652bdf2' server FQDN: '61f7c652bdf2'
SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false
SASL_AUXPROP_AUTHZID: false
I1109 19:00:52.897600 29637 auxprop.cpp:181] Looking up auxiliary property
'*userPassword'
I1109 19:00:52.897655 29637 auxprop.cpp:181] Looking up auxiliary property
'*cmusaslsecretCRAM-MD5'
I1109 19:00:52.897689 29637 auxprop.cpp:109] Request to lookup properties for
user: 'test-principal' realm: '61f7c652bdf2' server FQDN: '61f7c652bdf2'
SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false
SASL_AUXPROP_AUTHZID: true
I1109 19:00:52.897703 29637 auxprop.cpp:131] Skipping auxiliary property
'*userPassword' since SASL_AUXPROP_AUTHZID == true
I1109 19:00:52.897713 29637 auxprop.cpp:131] Skipping auxiliary property
'*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true
I1109 19:00:52.897733 29637 authenticator.cpp:319] Authentication success
I1109 19:00:52.897814 29626 authenticatee.cpp:300] Authentication success
I1109 19:00:52.898047 29625 sched.cpp:409] Successfully authenticated with
master [email protected]:60889
I1109 19:00:52.898072 29625 sched.cpp:716] Sending SUBSCRIBE call to
[email protected]:60889
I1109 19:00:52.898152 29625 sched.cpp:749] Will retry registration in
663.938562ms if necessary
I1109 19:00:52.898216 29637 master.cpp:5180] Successfully authenticated
principal 'test-principal' at
[email protected]:60889
I1109 19:00:52.898265 29625 authenticator.cpp:433] Authentication session
cleanup for crammd5_authenticatee(164)@172.17.14.49:60889
I1109 19:00:52.898396 29637 master.cpp:2176] Received SUBSCRIBE call for
framework 'default' at
[email protected]:60889
I1109 19:00:52.898465 29637 master.cpp:1645] Authorizing framework principal
'test-principal' to receive offers for role '*'
I1109 19:00:52.898475 29625 slave.cpp:4230] Finished recovery
I1109 19:00:52.898919 29637 master.cpp:2247] Subscribing framework default with
checkpointing disabled and capabilities [ ]
I1109 19:00:52.899102 29625 slave.cpp:4387] Querying resource estimator for
oversubscribable resources
I1109 19:00:52.899441 29625 hierarchical.cpp:185] Added framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000
I1109 19:00:52.899560 29625 hierarchical.cpp:971] No resources available to
allocate!
I1109 19:00:52.899654 29632 status_update_manager.cpp:176] Pausing sending
status updates
I1109 19:00:52.899659 29637 sched.cpp:643] Framework registered with
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000
I1109 19:00:52.899585 29633 slave.cpp:729] New master detected at
[email protected]:60889
I1109 19:00:52.899716 29637 sched.cpp:657] Scheduler::registered took 22789ns
I1109 19:00:52.899736 29633 slave.cpp:792] Authenticating with master
[email protected]:60889
I1109 19:00:52.899755 29633 slave.cpp:797] Using default CRAM-MD5 authenticatee
I1109 19:00:52.899827 29625 hierarchical.cpp:1064] No inverse offers to send
out!
I1109 19:00:52.899948 29625 hierarchical.cpp:870] Performed allocation for 0
slaves in 424469ns
I1109 19:00:52.899956 29634 authenticatee.cpp:123] Creating new client SASL
connection
I1109 19:00:52.899889 29633 slave.cpp:765] Detecting new master
I1109 19:00:52.900290 29623 master.cpp:5150] Authenticating
slave(59)@172.17.14.49:60889
I1109 19:00:52.900306 29633 slave.cpp:4401] Received oversubscribable resources
from the resource estimator
I1109 19:00:52.900372 29634 authenticator.cpp:415] Starting authentication
session for crammd5_authenticatee(165)@172.17.14.49:60889
I1109 19:00:52.900554 29622 authenticator.cpp:100] Creating new server SASL
connection
I1109 19:00:52.900802 29622 authenticatee.cpp:214] Received SASL authentication
mechanisms: CRAM-MD5
I1109 19:00:52.900884 29622 authenticatee.cpp:240] Attempting to authenticate
with mechanism 'CRAM-MD5'
I1109 19:00:52.901053 29622 authenticator.cpp:205] Received SASL authentication
start
I1109 19:00:52.901101 29622 authenticator.cpp:327] Authentication requires more
steps
I1109 19:00:52.901276 29622 authenticatee.cpp:260] Received SASL authentication
step
I1109 19:00:52.901412 29622 authenticator.cpp:233] Received SASL authentication
step
I1109 19:00:52.901515 29622 auxprop.cpp:109] Request to lookup properties for
user: 'test-principal' realm: '61f7c652bdf2' server FQDN: '61f7c652bdf2'
SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false
SASL_AUXPROP_AUTHZID: false
I1109 19:00:52.901583 29622 auxprop.cpp:181] Looking up auxiliary property
'*userPassword'
I1109 19:00:52.901626 29622 auxprop.cpp:181] Looking up auxiliary property
'*cmusaslsecretCRAM-MD5'
I1109 19:00:52.901651 29622 auxprop.cpp:109] Request to lookup properties for
user: 'test-principal' realm: '61f7c652bdf2' server FQDN: '61f7c652bdf2'
SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false
SASL_AUXPROP_AUTHZID: true
I1109 19:00:52.901662 29622 auxprop.cpp:131] Skipping auxiliary property
'*userPassword' since SASL_AUXPROP_AUTHZID == true
I1109 19:00:52.901670 29622 auxprop.cpp:131] Skipping auxiliary property
'*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true
I1109 19:00:52.901686 29622 authenticator.cpp:319] Authentication success
I1109 19:00:52.901823 29622 authenticatee.cpp:300] Authentication success
I1109 19:00:52.902007 29622 master.cpp:5180] Successfully authenticated
principal 'test-principal' at slave(59)@172.17.14.49:60889
I1109 19:00:52.902092 29622 authenticator.cpp:433] Authentication session
cleanup for crammd5_authenticatee(165)@172.17.14.49:60889
I1109 19:00:52.902946 29633 slave.cpp:860] Successfully authenticated with
master [email protected]:60889
I1109 19:00:52.903100 29633 slave.cpp:1254] Will retry registration in 492605ns
if necessary
I1109 19:00:52.903314 29627 master.cpp:3859] Registering slave at
slave(59)@172.17.14.49:60889 (61f7c652bdf2) with id
34506b61-7fc1-4e5e-821c-e5f5650782e0-S0
I1109 19:00:52.903866 29629 registrar.cpp:441] Applied 1 operations in 68516ns;
attempting to update the 'registry'
I1109 19:00:52.904614 29633 slave.cpp:1254] Will retry registration in 839715ns
if necessary
I1109 19:00:52.904727 29636 log.cpp:685] Attempting to append 345 bytes to the
log
I1109 19:00:52.904736 29623 master.cpp:3847] Ignoring register slave message
from slave(59)@172.17.14.49:60889 (61f7c652bdf2) as admission is already in
progress
I1109 19:00:52.904885 29634 coordinator.cpp:350] Coordinator attempting to
write APPEND action at position 3
I1109 19:00:52.905652 29633 replica.cpp:540] Replica received write request for
position 3 from (1882)@172.17.14.49:60889
I1109 19:00:52.906004 29624 slave.cpp:1254] Will retry registration in
50.877167ms if necessary
I1109 19:00:52.906114 29629 master.cpp:3847] Ignoring register slave message
from slave(59)@172.17.14.49:60889 (61f7c652bdf2) as admission is already in
progress
I1109 19:00:52.957790 29623 slave.cpp:1254] Will retry registration in
148.906602ms if necessary
I1109 19:00:52.958108 29623 master.cpp:3847] Ignoring register slave message
from slave(59)@172.17.14.49:60889 (61f7c652bdf2) as admission is already in
progress
I1109 19:00:52.963148 29633 leveldb.cpp:343] Persisting action (364 bytes) to
leveldb took 57.434275ms
I1109 19:00:52.963251 29633 replica.cpp:715] Persisted action at 3
I1109 19:00:52.964395 29623 replica.cpp:694] Replica received learned notice
for position 3 from @0.0.0.0:0
I1109 19:00:53.025077 29623 leveldb.cpp:343] Persisting action (366 bytes) to
leveldb took 60.616045ms
I1109 19:00:53.025174 29623 replica.cpp:715] Persisted action at 3
I1109 19:00:53.025214 29623 replica.cpp:700] Replica learned APPEND action at
position 3
I1109 19:00:53.027107 29624 registrar.cpp:486] Successfully updated the
'registry' in 123.156224ms
I1109 19:00:53.027403 29636 log.cpp:704] Attempting to truncate the log to 3
I1109 19:00:53.027645 29636 coordinator.cpp:350] Coordinator attempting to
write TRUNCATE action at position 4
I1109 19:00:53.028303 29625 slave.cpp:3169] Received ping from
slave-observer(56)@172.17.14.49:60889
I1109 19:00:53.028708 29627 replica.cpp:540] Replica received write request for
position 4 from (1883)@172.17.14.49:60889
I1109 19:00:53.028769 29631 master.cpp:3927] Registered slave
34506b61-7fc1-4e5e-821c-e5f5650782e0-S0 at slave(59)@172.17.14.49:60889
(61f7c652bdf2) with cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]
I1109 19:00:53.028837 29623 slave.cpp:904] Registered with master
[email protected]:60889; given slave ID
34506b61-7fc1-4e5e-821c-e5f5650782e0-S0
I1109 19:00:53.028861 29623 fetcher.cpp:79] Clearing fetcher cache
I1109 19:00:53.028920 29625 hierarchical.cpp:335] Added slave
34506b61-7fc1-4e5e-821c-e5f5650782e0-S0 (61f7c652bdf2) with cpus(*):2;
mem(*):1024; disk(*):1024; ports(*):[31000-32000] (allocated: )
I1109 19:00:53.029285 29623 slave.cpp:927] Checkpointing SlaveInfo to
'/tmp/HealthCheckTest_HealthStatusChange_xdqiRg/meta/slaves/34506b61-7fc1-4e5e-821c-e5f5650782e0-S0/slave.info'
I1109 19:00:53.029700 29623 slave.cpp:963] Forwarding total oversubscribed
resources
I1109 19:00:53.029793 29623 status_update_manager.cpp:183] Resuming sending
status updates
I1109 19:00:53.029832 29625 hierarchical.cpp:1064] No inverse offers to send
out!
I1109 19:00:53.029875 29625 hierarchical.cpp:886] Performed allocation for
slave 34506b61-7fc1-4e5e-821c-e5f5650782e0-S0 in 916734ns
I1109 19:00:53.029930 29623 master.cpp:4269] Received update of slave
34506b61-7fc1-4e5e-821c-e5f5650782e0-S0 at slave(59)@172.17.14.49:60889
(61f7c652bdf2) with total oversubscribed resources
I1109 19:00:53.030433 29625 hierarchical.cpp:391] Slave
34506b61-7fc1-4e5e-821c-e5f5650782e0-S0 (61f7c652bdf2) updated with
oversubscribed resources (total: cpus(*):2; mem(*):1024; disk(*):1024;
ports(*):[31000-32000], allocated: cpus(*):2; mem(*):1024; disk(*):1024;
ports(*):[31000-32000])
I1109 19:00:53.030494 29623 master.cpp:4979] Sending 1 offers to framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 (default) at
[email protected]:60889
I1109 19:00:53.030859 29625 hierarchical.cpp:971] No resources available to
allocate!
I1109 19:00:53.030895 29625 hierarchical.cpp:1064] No inverse offers to send
out!
I1109 19:00:53.030910 29625 hierarchical.cpp:886] Performed allocation for
slave 34506b61-7fc1-4e5e-821c-e5f5650782e0-S0 in 427184ns
I1109 19:00:53.031009 29634 sched.cpp:813] Scheduler::resourceOffers took
155158ns
I1109 19:00:53.032750 29623 master.cpp:2915] Processing ACCEPT call for offers:
[ 34506b61-7fc1-4e5e-821c-e5f5650782e0-O0 ] on slave
34506b61-7fc1-4e5e-821c-e5f5650782e0-S0 at slave(59)@172.17.14.49:60889
(61f7c652bdf2) for framework 34506b61-7fc1-4e5e-821c-e5f5650782e0-0000
(default) at [email protected]:60889
I1109 19:00:53.032804 29623 master.cpp:2711] Authorizing framework principal
'test-principal' to launch task 1 as user 'mesos'
I1109 19:00:53.034595 29635 master.hpp:176] Adding task 1 with resources
cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] on slave
34506b61-7fc1-4e5e-821c-e5f5650782e0-S0 (61f7c652bdf2)
I1109 19:00:53.034788 29635 master.cpp:3245] Launching task 1 of framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 (default) at
[email protected]:60889 with
resources cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] on slave
34506b61-7fc1-4e5e-821c-e5f5650782e0-S0 at slave(59)@172.17.14.49:60889
(61f7c652bdf2)
I1109 19:00:53.035286 29622 slave.cpp:1294] Got assigned task 1 for framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000
I1109 19:00:53.036186 29622 slave.cpp:1410] Launching task 1 for framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000
I1109 19:00:53.036998 29622 paths.cpp:417] Trying to chown
'/tmp/HealthCheckTest_HealthStatusChange_xdqiRg/slaves/34506b61-7fc1-4e5e-821c-e5f5650782e0-S0/frameworks/34506b61-7fc1-4e5e-821c-e5f5650782e0-0000/executors/1/runs/fa569209-1433-470c-ba09-34756185229a'
to user 'mesos'
I1109 19:00:53.040652 29622 slave.cpp:4999] Launching executor 1 of framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 with resources cpus(*):0.1; mem(*):32
in work directory
'/tmp/HealthCheckTest_HealthStatusChange_xdqiRg/slaves/34506b61-7fc1-4e5e-821c-e5f5650782e0-S0/frameworks/34506b61-7fc1-4e5e-821c-e5f5650782e0-0000/executors/1/runs/fa569209-1433-470c-ba09-34756185229a'
I1109 19:00:53.041373 29628 containerizer.cpp:618] Starting container
'fa569209-1433-470c-ba09-34756185229a' for executor '1' of framework
'34506b61-7fc1-4e5e-821c-e5f5650782e0-0000'
I1109 19:00:53.041410 29622 slave.cpp:1628] Queuing task '1' for executor ''1'
of framework 34506b61-7fc1-4e5e-821c-e5f5650782e0-0000'
I1109 19:00:53.041533 29622 slave.cpp:682] Successfully attached file
'/tmp/HealthCheckTest_HealthStatusChange_xdqiRg/slaves/34506b61-7fc1-4e5e-821c-e5f5650782e0-S0/frameworks/34506b61-7fc1-4e5e-821c-e5f5650782e0-0000/executors/1/runs/fa569209-1433-470c-ba09-34756185229a'
I1109 19:00:53.053426 29630 launcher.cpp:134] Forked child with pid '2245' for
container 'fa569209-1433-470c-ba09-34756185229a'
I1109 19:00:53.059900 29627 leveldb.cpp:343] Persisting action (16 bytes) to
leveldb took 31.125935ms
I1109 19:00:53.059988 29627 replica.cpp:715] Persisted action at 4
I1109 19:00:53.061278 29635 replica.cpp:694] Replica received learned notice
for position 4 from @0.0.0.0:0
I1109 19:00:53.110086 29635 leveldb.cpp:343] Persisting action (18 bytes) to
leveldb took 48.75579ms
I1109 19:00:53.110260 29635 leveldb.cpp:401] Deleting ~2 keys from leveldb took
89854ns
I1109 19:00:53.110287 29635 replica.cpp:715] Persisted action at 4
I1109 19:00:53.110322 29635 replica.cpp:700] Replica learned TRUNCATE action at
position 4
I1109 19:00:53.202918 29631 hierarchical.cpp:971] No resources available to
allocate!
I1109 19:00:53.202997 29631 hierarchical.cpp:1064] No inverse offers to send
out!
I1109 19:00:53.203024 29631 hierarchical.cpp:870] Performed allocation for 1
slaves in 598550ns
I1109 19:00:53.227322 29629 slave.cpp:2405] Got registration for executor '1'
of framework 34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 from
executor(1)@172.17.14.49:60202
I1109 19:00:53.230358 29633 slave.cpp:1793] Sending queued task '1' to executor
''1' of framework 34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 at
executor(1)@172.17.14.49:60202'
I1109 19:00:53.392653 29636 containerizer.cpp:1257] Executor for container
'fa569209-1433-470c-ba09-34756185229a' has exited
I1109 19:00:53.392715 29636 containerizer.cpp:1074] Destroying container
'fa569209-1433-470c-ba09-34756185229a'
I1109 19:00:53.396560 29635 slave.cpp:3553] Executor '1' of framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 exited with status 134
I1109 19:00:53.396731 29635 slave.cpp:2762] Handling status update TASK_FAILED
(UUID: ac5509fc-0172-4dca-9d66-1a76810277ae) for task 1 of framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 from @0.0.0.0:0
I1109 19:00:53.396927 29635 slave.cpp:5298] Terminating task 1
W1109 19:00:53.397447 29628 containerizer.cpp:966] Ignoring update for unknown
container: fa569209-1433-470c-ba09-34756185229a
I1109 19:00:53.397917 29627 status_update_manager.cpp:322] Received status
update TASK_FAILED (UUID: ac5509fc-0172-4dca-9d66-1a76810277ae) for task 1 of
framework 34506b61-7fc1-4e5e-821c-e5f5650782e0-0000
I1109 19:00:53.397963 29627 status_update_manager.cpp:499] Creating
StatusUpdate stream for task 1 of framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000
I1109 19:00:53.398464 29627 status_update_manager.cpp:376] Forwarding update
TASK_FAILED (UUID: ac5509fc-0172-4dca-9d66-1a76810277ae) for task 1 of
framework 34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 to the slave
I1109 19:00:53.398849 29628 slave.cpp:3087] Forwarding the update TASK_FAILED
(UUID: ac5509fc-0172-4dca-9d66-1a76810277ae) for task 1 of framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 to [email protected]:60889
I1109 19:00:53.399086 29628 slave.cpp:3005] Status update manager successfully
handled status update TASK_FAILED (UUID: ac5509fc-0172-4dca-9d66-1a76810277ae)
for task 1 of framework 34506b61-7fc1-4e5e-821c-e5f5650782e0-0000
I1109 19:00:53.399317 29623 master.cpp:4414] Status update TASK_FAILED (UUID:
ac5509fc-0172-4dca-9d66-1a76810277ae) for task 1 of framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 from slave
34506b61-7fc1-4e5e-821c-e5f5650782e0-S0 at slave(59)@172.17.14.49:60889
(61f7c652bdf2)
I1109 19:00:53.399370 29623 master.cpp:4462] Forwarding status update
TASK_FAILED (UUID: ac5509fc-0172-4dca-9d66-1a76810277ae) for task 1 of
framework 34506b61-7fc1-4e5e-821c-e5f5650782e0-0000
I1109 19:00:53.399559 29623 master.cpp:6066] Updating the state of task 1 of
framework 34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 (latest state: TASK_FAILED,
status update state: TASK_FAILED)
I1109 19:00:53.399818 29628 sched.cpp:921] Scheduler::statusUpdate took 144121ns
../../src/tests/health_check_tests.cpp:480: Failure
Value of: statusRunning.get().state()
Actual: TASK_FAILED
Expected: TASK_RUNNING
I1109 19:00:53.400254 29626 hierarchical.cpp:739] Recovered cpus(*):2;
mem(*):1024; disk(*):1024; ports(*):[31000-32000] (total: cpus(*):2;
mem(*):1024; disk(*):1024; ports(*):[31000-32000], allocated: ) on slave
34506b61-7fc1-4e5e-821c-e5f5650782e0-S0 from framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000
I1109 19:00:53.400329 29623 master.cpp:3571] Processing ACKNOWLEDGE call
ac5509fc-0172-4dca-9d66-1a76810277ae for task 1 of framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 (default) at
[email protected]:60889 on slave
34506b61-7fc1-4e5e-821c-e5f5650782e0-S0
I1109 19:00:53.400390 29623 master.cpp:6132] Removing task 1 with resources
cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] of framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 on slave
34506b61-7fc1-4e5e-821c-e5f5650782e0-S0 at slave(59)@172.17.14.49:60889
(61f7c652bdf2)
I1109 19:00:53.400840 29626 status_update_manager.cpp:394] Received status
update acknowledgement (UUID: ac5509fc-0172-4dca-9d66-1a76810277ae) for task 1
of framework 34506b61-7fc1-4e5e-821c-e5f5650782e0-0000
I1109 19:00:53.401018 29626 status_update_manager.cpp:530] Cleaning up status
update stream for task 1 of framework 34506b61-7fc1-4e5e-821c-e5f5650782e0-0000
I1109 19:00:53.401381 29633 slave.cpp:2345] Status update manager successfully
handled status update acknowledgement (UUID:
ac5509fc-0172-4dca-9d66-1a76810277ae) for task 1 of framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000
I1109 19:00:53.401428 29633 slave.cpp:5339] Completing task 1
I1109 19:00:53.401460 29633 slave.cpp:3657] Cleaning up executor '1' of
framework 34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 at
executor(1)@172.17.14.49:60202
I1109 19:00:53.401671 29631 gc.cpp:56] Scheduling
'/tmp/HealthCheckTest_HealthStatusChange_xdqiRg/slaves/34506b61-7fc1-4e5e-821c-e5f5650782e0-S0/frameworks/34506b61-7fc1-4e5e-821c-e5f5650782e0-0000/executors/1/runs/fa569209-1433-470c-ba09-34756185229a'
for gc 6.99999535184days in the future
I1109 19:00:53.401804 29633 slave.cpp:3745] Cleaning up framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000
I1109 19:00:53.401814 29631 gc.cpp:56] Scheduling
'/tmp/HealthCheckTest_HealthStatusChange_xdqiRg/slaves/34506b61-7fc1-4e5e-821c-e5f5650782e0-S0/frameworks/34506b61-7fc1-4e5e-821c-e5f5650782e0-0000/executors/1'
for gc 6.99999535011556days in the future
I1109 19:00:53.401914 29627 status_update_manager.cpp:284] Closing status
update streams for framework 34506b61-7fc1-4e5e-821c-e5f5650782e0-0000
I1109 19:00:53.402007 29630 gc.cpp:56] Scheduling
'/tmp/HealthCheckTest_HealthStatusChange_xdqiRg/slaves/34506b61-7fc1-4e5e-821c-e5f5650782e0-S0/frameworks/34506b61-7fc1-4e5e-821c-e5f5650782e0-0000'
for gc 6.99999534828741days in the future
I1109 19:00:54.205024 29629 hierarchical.cpp:1064] No inverse offers to send
out!
I1109 19:00:54.205111 29629 hierarchical.cpp:870] Performed allocation for 1
slaves in 1.036657ms
I1109 19:00:54.205626 29636 master.cpp:4979] Sending 1 offers to framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 (default) at
[email protected]:60889
I1109 19:00:54.206218 29628 sched.cpp:813] Scheduler::resourceOffers took
27139ns
I1109 19:00:55.206257 29636 hierarchical.cpp:971] No resources available to
allocate!
I1109 19:00:55.206348 29636 hierarchical.cpp:1064] No inverse offers to send
out!
I1109 19:00:55.206382 29636 hierarchical.cpp:870] Performed allocation for 1
slaves in 709301ns
I1109 19:00:56.207715 29625 hierarchical.cpp:971] No resources available to
allocate!
I1109 19:00:56.207803 29625 hierarchical.cpp:1064] No inverse offers to send
out!
I1109 19:00:56.207834 29625 hierarchical.cpp:870] Performed allocation for 1
slaves in 707048ns
I1109 19:00:57.209532 29626 hierarchical.cpp:971] No resources available to
allocate!
I1109 19:00:57.209610 29626 hierarchical.cpp:1064] No inverse offers to send
out!
I1109 19:00:57.209638 29626 hierarchical.cpp:870] Performed allocation for 1
slaves in 601219ns
I1109 19:00:58.211696 29628 hierarchical.cpp:971] No resources available to
allocate!
I1109 19:00:58.211796 29628 hierarchical.cpp:1064] No inverse offers to send
out!
I1109 19:00:58.211830 29628 hierarchical.cpp:870] Performed allocation for 1
slaves in 784177ns
I1109 19:00:59.212745 29622 hierarchical.cpp:971] No resources available to
allocate!
I1109 19:00:59.212826 29622 hierarchical.cpp:1064] No inverse offers to send
out!
I1109 19:00:59.212854 29622 hierarchical.cpp:870] Performed allocation for 1
slaves in 669442ns
I1109 19:01:00.214867 29637 hierarchical.cpp:971] No resources available to
allocate!
I1109 19:01:00.214956 29637 hierarchical.cpp:1064] No inverse offers to send
out!
I1109 19:01:00.214992 29637 hierarchical.cpp:870] Performed allocation for 1
slaves in 736461ns
I1109 19:01:01.216673 29634 hierarchical.cpp:971] No resources available to
allocate!
I1109 19:01:01.216763 29634 hierarchical.cpp:1064] No inverse offers to send
out!
I1109 19:01:01.216800 29634 hierarchical.cpp:870] Performed allocation for 1
slaves in 794520ns
I1109 19:01:02.218780 29634 hierarchical.cpp:971] No resources available to
allocate!
I1109 19:01:02.218878 29634 hierarchical.cpp:1064] No inverse offers to send
out!
I1109 19:01:02.218919 29634 hierarchical.cpp:870] Performed allocation for 1
slaves in 800091ns
I1109 19:01:03.220408 29630 hierarchical.cpp:971] No resources available to
allocate!
I1109 19:01:03.220509 29630 hierarchical.cpp:1064] No inverse offers to send
out!
I1109 19:01:03.220541 29630 hierarchical.cpp:870] Performed allocation for 1
slaves in 793742ns
I1109 19:01:04.222898 29637 hierarchical.cpp:971] No resources available to
allocate!
I1109 19:01:04.223003 29637 hierarchical.cpp:1064] No inverse offers to send
out!
I1109 19:01:04.223045 29637 hierarchical.cpp:870] Performed allocation for 1
slaves in 885294ns
I1109 19:01:05.224395 29631 hierarchical.cpp:971] No resources available to
allocate!
I1109 19:01:05.224472 29631 hierarchical.cpp:1064] No inverse offers to send
out!
I1109 19:01:05.224503 29631 hierarchical.cpp:870] Performed allocation for 1
slaves in 685482ns
I1109 19:01:06.225836 29626 hierarchical.cpp:971] No resources available to
allocate!
I1109 19:01:06.225935 29626 hierarchical.cpp:1064] No inverse offers to send
out!
I1109 19:01:06.225970 29626 hierarchical.cpp:870] Performed allocation for 1
slaves in 855878ns
I1109 19:01:07.227568 29624 hierarchical.cpp:971] No resources available to
allocate!
I1109 19:01:07.227658 29624 hierarchical.cpp:1064] No inverse offers to send
out!
I1109 19:01:07.227699 29624 hierarchical.cpp:870] Performed allocation for 1
slaves in 667674ns
I1109 19:01:07.901265 29625 slave.cpp:4387] Querying resource estimator for
oversubscribable resources
I1109 19:01:07.901639 29635 slave.cpp:4401] Received oversubscribable resources
from the resource estimator
I1109 19:01:08.029749 29637 slave.cpp:3169] Received ping from
slave-observer(56)@172.17.14.49:60889
I1109 19:01:08.229215 29624 hierarchical.cpp:971] No resources available to
allocate!
I1109 19:01:08.229305 29624 hierarchical.cpp:1064] No inverse offers to send
out!
I1109 19:01:08.229336 29624 hierarchical.cpp:870] Performed allocation for 1
slaves in 730966ns
../../src/tests/health_check_tests.cpp:482: Failure
Failed to wait 15secs for statusHealth1
../../src/tests/health_check_tests.cpp:471: Failure
Actual function call count doesn't match EXPECT_CALL(sched,
statusUpdate(&driver, _))...
Expected: to be called 4 times
Actual: called once - unsatisfied and active
I1109 19:01:08.404537 29636 master.cpp:1122] Framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 (default) at
[email protected]:60889 disconnected
I1109 19:01:08.405421 29636 master.cpp:2472] Disconnecting framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 (default) at
[email protected]:60889
I1109 19:01:08.405673 29636 master.cpp:2496] Deactivating framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 (default) at
[email protected]:60889
I1109 19:01:08.406396 29627 hierarchical.cpp:263] Deactivated framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000
W1109 19:01:08.407588 29636 master.hpp:1540] Master attempted to send message
to disconnected framework 34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 (default)
at [email protected]:60889
I1109 19:01:08.407769 29636 master.cpp:1146] Giving framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 (default) at
[email protected]:60889 0ns to
failover
I1109 19:01:08.407088 29627 hierarchical.cpp:739] Recovered cpus(*):2;
mem(*):1024; disk(*):1024; ports(*):[31000-32000] (total: cpus(*):2;
mem(*):1024; disk(*):1024; ports(*):[31000-32000], allocated: ) on slave
34506b61-7fc1-4e5e-821c-e5f5650782e0-S0 from framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000
I1109 19:01:08.410311 29633 master.cpp:4827] Framework failover timeout,
removing framework 34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 (default) at
[email protected]:60889
I1109 19:01:08.410362 29633 master.cpp:5559] Removing framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 (default) at
[email protected]:60889
I1109 19:01:08.410769 29633 slave.cpp:2009] Asked to shut down framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000 by [email protected]:60889
W1109 19:01:08.410820 29633 slave.cpp:2024] Cannot shut down unknown framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000
I1109 19:01:08.411039 29633 hierarchical.cpp:220] Removed framework
34506b61-7fc1-4e5e-821c-e5f5650782e0-0000
I1109 19:01:08.411375 29603 master.cpp:922] Master terminating
I1109 19:01:08.412467 29633 hierarchical.cpp:364] Removed slave
34506b61-7fc1-4e5e-821c-e5f5650782e0-S0
I1109 19:01:08.412521 29633 slave.cpp:3215] [email protected]:60889 exited
W1109 19:01:08.412542 29633 slave.cpp:3218] Master disconnected! Waiting for a
new master to be elected
I1109 19:01:08.430963 29637 slave.cpp:601] Slave terminating
[ FAILED ] HealthCheckTest.HealthStatusChange (16622 ms)
{code}
> HealthCheckTest.HealthStatusChange is flaky on jenkins.
> -------------------------------------------------------
>
> Key: MESOS-1802
> URL: https://issues.apache.org/jira/browse/MESOS-1802
> Project: Mesos
> Issue Type: Bug
> Components: test
> Reporter: Benjamin Mahler
> Assignee: Timothy Chen
> Labels: flaky, mesosphere
>
> https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Disable-Java-Disable-Python-Disable-Webui/2374/consoleFull
> {noformat}
> [ RUN ] HealthCheckTest.HealthStatusChange
> Using temporary directory '/tmp/HealthCheckTest_HealthStatusChange_IYnlu2'
> I0916 22:56:14.034612 21026 leveldb.cpp:176] Opened db in 2.155713ms
> I0916 22:56:14.034965 21026 leveldb.cpp:183] Compacted db in 332489ns
> I0916 22:56:14.034984 21026 leveldb.cpp:198] Created db iterator in 3710ns
> I0916 22:56:14.034996 21026 leveldb.cpp:204] Seeked to beginning of db in
> 642ns
> I0916 22:56:14.035006 21026 leveldb.cpp:273] Iterated through 0 keys in the
> db in 343ns
> I0916 22:56:14.035023 21026 replica.cpp:741] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I0916 22:56:14.035200 21054 recover.cpp:425] Starting replica recovery
> I0916 22:56:14.035403 21041 recover.cpp:451] Replica is in EMPTY status
> I0916 22:56:14.035888 21045 replica.cpp:638] Replica in EMPTY status received
> a broadcasted recover request
> I0916 22:56:14.035969 21052 recover.cpp:188] Received a recover response from
> a replica in EMPTY status
> I0916 22:56:14.036118 21042 recover.cpp:542] Updating replica status to
> STARTING
> I0916 22:56:14.036603 21046 master.cpp:286] Master
> 20140916-225614-3125920579-47865-21026 (penates.apache.org) started on
> 67.195.81.186:47865
> I0916 22:56:14.036634 21046 master.cpp:332] Master only allowing
> authenticated frameworks to register
> I0916 22:56:14.036648 21046 master.cpp:337] Master only allowing
> authenticated slaves to register
> I0916 22:56:14.036659 21046 credentials.hpp:36] Loading credentials for
> authentication from
> '/tmp/HealthCheckTest_HealthStatusChange_IYnlu2/credentials'
> I0916 22:56:14.036686 21045 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 480322ns
> I0916 22:56:14.036700 21045 replica.cpp:320] Persisted replica status to
> STARTING
> I0916 22:56:14.036769 21046 master.cpp:366] Authorization enabled
> I0916 22:56:14.036826 21045 recover.cpp:451] Replica is in STARTING status
> I0916 22:56:14.036944 21052 master.cpp:120] No whitelist given. Advertising
> offers for all slaves
> I0916 22:56:14.036968 21049 hierarchical_allocator_process.hpp:299]
> Initializing hierarchical allocator process with master :
> [email protected]:47865
> I0916 22:56:14.037284 21054 replica.cpp:638] Replica in STARTING status
> received a broadcasted recover request
> I0916 22:56:14.037312 21046 master.cpp:1212] The newly elected leader is
> [email protected]:47865 with id 20140916-225614-3125920579-47865-21026
> I0916 22:56:14.037333 21046 master.cpp:1225] Elected as the leading master!
> I0916 22:56:14.037345 21046 master.cpp:1043] Recovering from registrar
> I0916 22:56:14.037504 21040 registrar.cpp:313] Recovering registrar
> I0916 22:56:14.037505 21053 recover.cpp:188] Received a recover response from
> a replica in STARTING status
> I0916 22:56:14.037681 21047 recover.cpp:542] Updating replica status to VOTING
> I0916 22:56:14.038072 21052 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 330251ns
> I0916 22:56:14.038087 21052 replica.cpp:320] Persisted replica status to
> VOTING
> I0916 22:56:14.038127 21053 recover.cpp:556] Successfully joined the Paxos
> group
> I0916 22:56:14.038202 21053 recover.cpp:440] Recover process terminated
> I0916 22:56:14.038364 21048 log.cpp:656] Attempting to start the writer
> I0916 22:56:14.038812 21053 replica.cpp:474] Replica received implicit
> promise request with proposal 1
> I0916 22:56:14.038925 21053 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 92623ns
> I0916 22:56:14.038944 21053 replica.cpp:342] Persisted promised to 1
> I0916 22:56:14.039201 21052 coordinator.cpp:230] Coordinator attemping to
> fill missing position
> I0916 22:56:14.039676 21047 replica.cpp:375] Replica received explicit
> promise request for position 0 with proposal 2
> I0916 22:56:14.039836 21047 leveldb.cpp:343] Persisting action (8 bytes) to
> leveldb took 144215ns
> I0916 22:56:14.039850 21047 replica.cpp:676] Persisted action at 0
> I0916 22:56:14.040243 21047 replica.cpp:508] Replica received write request
> for position 0
> I0916 22:56:14.040267 21047 leveldb.cpp:438] Reading position from leveldb
> took 10323ns
> I0916 22:56:14.040362 21047 leveldb.cpp:343] Persisting action (14 bytes) to
> leveldb took 79471ns
> I0916 22:56:14.040375 21047 replica.cpp:676] Persisted action at 0
> I0916 22:56:14.040556 21054 replica.cpp:655] Replica received learned notice
> for position 0
> I0916 22:56:14.040658 21054 leveldb.cpp:343] Persisting action (16 bytes) to
> leveldb took 83975ns
> I0916 22:56:14.040676 21054 replica.cpp:676] Persisted action at 0
> I0916 22:56:14.040689 21054 replica.cpp:661] Replica learned NOP action at
> position 0
> I0916 22:56:14.041023 21043 log.cpp:672] Writer started with ending position 0
> I0916 22:56:14.041342 21052 leveldb.cpp:438] Reading position from leveldb
> took 10642ns
> I0916 22:56:14.042325 21050 registrar.cpp:346] Successfully fetched the
> registry (0B)
> I0916 22:56:14.042346 21050 registrar.cpp:422] Attempting to update the
> 'registry'
> I0916 22:56:14.043306 21054 log.cpp:680] Attempting to append 140 bytes to
> the log
> I0916 22:56:14.043354 21050 coordinator.cpp:340] Coordinator attempting to
> write APPEND action at position 1
> I0916 22:56:14.043637 21047 replica.cpp:508] Replica received write request
> for position 1
> I0916 22:56:14.044042 21047 leveldb.cpp:343] Persisting action (159 bytes) to
> leveldb took 386690ns
> I0916 22:56:14.044057 21047 replica.cpp:676] Persisted action at 1
> I0916 22:56:14.044271 21040 replica.cpp:655] Replica received learned notice
> for position 1
> I0916 22:56:14.044435 21040 leveldb.cpp:343] Persisting action (161 bytes) to
> leveldb took 145186ns
> I0916 22:56:14.044448 21040 replica.cpp:676] Persisted action at 1
> I0916 22:56:14.044456 21040 replica.cpp:661] Replica learned APPEND action at
> position 1
> I0916 22:56:14.044729 21055 registrar.cpp:479] Successfully updated 'registry'
> I0916 22:56:14.044776 21047 log.cpp:699] Attempting to truncate the log to 1
> I0916 22:56:14.044795 21055 registrar.cpp:372] Successfully recovered
> registrar
> I0916 22:56:14.044831 21051 coordinator.cpp:340] Coordinator attempting to
> write TRUNCATE action at position 2
> I0916 22:56:14.044899 21053 master.cpp:1070] Recovered 0 slaves from the
> Registry (102B) ; allowing 10mins for slaves to re-register
> I0916 22:56:14.045133 21055 replica.cpp:508] Replica received write request
> for position 2
> I0916 22:56:14.045450 21055 leveldb.cpp:343] Persisting action (16 bytes) to
> leveldb took 300867ns
> I0916 22:56:14.045465 21055 replica.cpp:676] Persisted action at 2
> I0916 22:56:14.045725 21052 replica.cpp:655] Replica received learned notice
> for position 2
> I0916 22:56:14.045925 21052 leveldb.cpp:343] Persisting action (18 bytes) to
> leveldb took 182657ns
> I0916 22:56:14.045948 21052 leveldb.cpp:401] Deleting ~1 keys from leveldb
> took 10733ns
> I0916 22:56:14.045958 21052 replica.cpp:676] Persisted action at 2
> I0916 22:56:14.045964 21052 replica.cpp:661] Replica learned TRUNCATE action
> at position 2
> I0916 22:56:14.055306 21026 containerizer.cpp:89] Using isolation:
> posix/cpu,posix/mem
> I0916 22:56:14.057139 21048 slave.cpp:169] Slave started on
> 102)@67.195.81.186:47865
> I0916 22:56:14.057178 21048 credentials.hpp:84] Loading credential for
> authentication from
> '/tmp/HealthCheckTest_HealthStatusChange_cGKTiG/credential'
> I0916 22:56:14.057283 21048 slave.cpp:276] Slave using credential for:
> test-principal
> I0916 22:56:14.057354 21048 slave.cpp:289] Slave resources: cpus(*):2;
> mem(*):1024; disk(*):1024; ports(*):[31000-32000]
> I0916 22:56:14.057457 21048 slave.cpp:317] Slave hostname: penates.apache.org
> I0916 22:56:14.057468 21048 slave.cpp:318] Slave checkpoint: false
> I0916 22:56:14.057754 21043 state.cpp:33] Recovering state from
> '/tmp/HealthCheckTest_HealthStatusChange_cGKTiG/meta'
> I0916 22:56:14.057864 21042 status_update_manager.cpp:193] Recovering status
> update manager
> I0916 22:56:14.057958 21042 containerizer.cpp:252] Recovering containerizer
> I0916 22:56:14.058226 21042 slave.cpp:3219] Finished recovery
> I0916 22:56:14.058452 21047 slave.cpp:600] New master detected at
> [email protected]:47865
> I0916 22:56:14.058485 21047 slave.cpp:674] Authenticating with master
> [email protected]:47865
> I0916 22:56:14.058506 21042 status_update_manager.cpp:167] New master
> detected at [email protected]:47865
> I0916 22:56:14.058539 21047 slave.cpp:647] Detecting new master
> I0916 22:56:14.058555 21042 authenticatee.hpp:128] Creating new client SASL
> connection
> I0916 22:56:14.058656 21043 master.cpp:3653] Authenticating
> slave(102)@67.195.81.186:47865
> I0916 22:56:14.058737 21040 authenticator.hpp:156] Creating new server SASL
> connection
> I0916 22:56:14.058830 21047 authenticatee.hpp:219] Received SASL
> authentication mechanisms: CRAM-MD5
> I0916 22:56:14.058852 21047 authenticatee.hpp:245] Attempting to authenticate
> with mechanism 'CRAM-MD5'
> I0916 22:56:14.058884 21047 authenticator.hpp:262] Received SASL
> authentication start
> I0916 22:56:14.058936 21047 authenticator.hpp:384] Authentication requires
> more steps
> I0916 22:56:14.058981 21047 authenticatee.hpp:265] Received SASL
> authentication step
> I0916 22:56:14.059052 21040 authenticator.hpp:290] Received SASL
> authentication step
> I0916 22:56:14.059074 21040 auxprop.cpp:81] Request to lookup properties for
> user: 'test-principal' realm: 'penates.apache.org' server FQDN:
> 'penates.apache.org' SASL_AUXPROP_VERIFY_AGAINST_HASH: false
> SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false
> I0916 22:56:14.059087 21040 auxprop.cpp:153] Looking up auxiliary property
> '*userPassword'
> I0916 22:56:14.059101 21040 auxprop.cpp:153] Looking up auxiliary property
> '*cmusaslsecretCRAM-MD5'
> I0916 22:56:14.059111 21040 auxprop.cpp:81] Request to lookup properties for
> user: 'test-principal' realm: 'penates.apache.org' server FQDN:
> 'penates.apache.org' SASL_AUXPROP_VERIFY_AGAINST_HASH: false
> SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true
> I0916 22:56:14.059118 21040 auxprop.cpp:103] Skipping auxiliary property
> '*userPassword' since SASL_AUXPROP_AUTHZID == true
> I0916 22:56:14.059123 21040 auxprop.cpp:103] Skipping auxiliary property
> '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true
> I0916 22:56:14.059135 21040 authenticator.hpp:376] Authentication success
> I0916 22:56:14.059182 21047 authenticatee.hpp:305] Authentication success
> I0916 22:56:14.059192 21040 master.cpp:3693] Successfully authenticated
> principal 'test-principal' at slave(102)@67.195.81.186:47865
> I0916 22:56:14.059309 21047 slave.cpp:731] Successfully authenticated with
> master [email protected]:47865
> I0916 22:56:14.059348 21047 slave.cpp:994] Will retry registration in
> 12.6149ms if necessary
> I0916 22:56:14.059396 21040 master.cpp:2843] Registering slave at
> slave(102)@67.195.81.186:47865 (penates.apache.org) with id
> 20140916-225614-3125920579-47865-21026-0
> I0916 22:56:14.059495 21054 registrar.cpp:422] Attempting to update the
> 'registry'
> I0916 22:56:14.059558 21026 sched.cpp:137] Version: 0.21.0
> I0916 22:56:14.059710 21041 sched.cpp:233] New master detected at
> [email protected]:47865
> I0916 22:56:14.059730 21041 sched.cpp:283] Authenticating with master
> [email protected]:47865
> I0916 22:56:14.059788 21052 authenticatee.hpp:128] Creating new client SASL
> connection
> I0916 22:56:14.059890 21043 master.cpp:3653] Authenticating
> [email protected]:47865
> I0916 22:56:14.059960 21055 authenticator.hpp:156] Creating new server SASL
> connection
> I0916 22:56:14.060039 21040 authenticatee.hpp:219] Received SASL
> authentication mechanisms: CRAM-MD5
> I0916 22:56:14.060061 21040 authenticatee.hpp:245] Attempting to authenticate
> with mechanism 'CRAM-MD5'
> I0916 22:56:14.060107 21055 authenticator.hpp:262] Received SASL
> authentication start
> I0916 22:56:14.060158 21055 authenticator.hpp:384] Authentication requires
> more steps
> I0916 22:56:14.060189 21055 authenticatee.hpp:265] Received SASL
> authentication step
> I0916 22:56:14.060220 21055 authenticator.hpp:290] Received SASL
> authentication step
> I0916 22:56:14.060236 21055 auxprop.cpp:81] Request to lookup properties for
> user: 'test-principal' realm: 'penates.apache.org' server FQDN:
> 'penates.apache.org' SASL_AUXPROP_VERIFY_AGAINST_HASH: false
> SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false
> I0916 22:56:14.060250 21055 auxprop.cpp:153] Looking up auxiliary property
> '*userPassword'
> I0916 22:56:14.060277 21055 auxprop.cpp:153] Looking up auxiliary property
> '*cmusaslsecretCRAM-MD5'
> I0916 22:56:14.060288 21055 auxprop.cpp:81] Request to lookup properties for
> user: 'test-principal' realm: 'penates.apache.org' server FQDN:
> 'penates.apache.org' SASL_AUXPROP_VERIFY_AGAINST_HASH: false
> SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true
> I0916 22:56:14.060295 21055 auxprop.cpp:103] Skipping auxiliary property
> '*userPassword' since SASL_AUXPROP_AUTHZID == true
> I0916 22:56:14.060300 21055 auxprop.cpp:103] Skipping auxiliary property
> '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true
> I0916 22:56:14.060312 21055 authenticator.hpp:376] Authentication success
> I0916 22:56:14.060349 21040 authenticatee.hpp:305] Authentication success
> I0916 22:56:14.060364 21055 master.cpp:3693] Successfully authenticated
> principal 'test-principal' at
> [email protected]:47865
> I0916 22:56:14.060480 21046 sched.cpp:357] Successfully authenticated with
> master [email protected]:47865
> I0916 22:56:14.060499 21046 sched.cpp:476] Sending registration request to
> [email protected]:47865
> I0916 22:56:14.060564 21050 master.cpp:1331] Received registration request
> from [email protected]:47865
> I0916 22:56:14.060593 21050 master.cpp:1291] Authorizing framework principal
> 'test-principal' to receive offers for role '*'
> I0916 22:56:14.060767 21053 master.cpp:1390] Registering framework
> 20140916-225614-3125920579-47865-21026-0000 at
> [email protected]:47865
> I0916 22:56:14.060797 21049 log.cpp:680] Attempting to append 337 bytes to
> the log
> I0916 22:56:14.060873 21042 hierarchical_allocator_process.hpp:329] Added
> framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:14.060873 21040 coordinator.cpp:340] Coordinator attempting to
> write APPEND action at position 3
> I0916 22:56:14.060899 21042 hierarchical_allocator_process.hpp:697] No
> resources available to allocate!
> I0916 22:56:14.060909 21042 hierarchical_allocator_process.hpp:659] Performed
> allocation for 0 slaves in 11862ns
> I0916 22:56:14.061061 21044 sched.cpp:407] Framework registered with
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:14.061115 21044 sched.cpp:421] Scheduler::registered took 34395ns
> I0916 22:56:14.061173 21047 replica.cpp:508] Replica received write request
> for position 3
> I0916 22:56:14.061298 21047 leveldb.cpp:343] Persisting action (356 bytes) to
> leveldb took 108843ns
> I0916 22:56:14.061311 21047 replica.cpp:676] Persisted action at 3
> I0916 22:56:14.061553 21049 replica.cpp:655] Replica received learned notice
> for position 3
> I0916 22:56:14.061965 21049 leveldb.cpp:343] Persisting action (358 bytes) to
> leveldb took 392670ns
> I0916 22:56:14.061985 21049 replica.cpp:676] Persisted action at 3
> I0916 22:56:14.061996 21049 replica.cpp:661] Replica learned APPEND action at
> position 3
> I0916 22:56:14.062268 21050 registrar.cpp:479] Successfully updated 'registry'
> I0916 22:56:14.062331 21051 log.cpp:699] Attempting to truncate the log to 3
> I0916 22:56:14.062355 21040 master.cpp:2883] Registered slave
> 20140916-225614-3125920579-47865-21026-0 at slave(102)@67.195.81.186:47865
> (penates.apache.org)
> I0916 22:56:14.062386 21043 coordinator.cpp:340] Coordinator attempting to
> write TRUNCATE action at position 4
> I0916 22:56:14.062376 21040 master.cpp:4126] Adding slave
> 20140916-225614-3125920579-47865-21026-0 at slave(102)@67.195.81.186:47865
> (penates.apache.org) with cpus(*):2; mem(*):1024; disk(*):1024;
> ports(*):[31000-32000]
> I0916 22:56:14.062510 21045 slave.cpp:765] Registered with master
> [email protected]:47865; given slave ID
> 20140916-225614-3125920579-47865-21026-0
> I0916 22:56:14.062573 21045 slave.cpp:2346] Received ping from
> slave-observer(98)@67.195.81.186:47865
> I0916 22:56:14.062599 21049 hierarchical_allocator_process.hpp:442] Added
> slave 20140916-225614-3125920579-47865-21026-0 (penates.apache.org) with
> cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] (and cpus(*):2;
> mem(*):1024; disk(*):1024; ports(*):[31000-32000] available)
> I0916 22:56:14.062669 21049 hierarchical_allocator_process.hpp:734] Offering
> cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] on slave
> 20140916-225614-3125920579-47865-21026-0 to framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:14.062764 21041 replica.cpp:508] Replica received write request
> for position 4
> I0916 22:56:14.062788 21049 hierarchical_allocator_process.hpp:679] Performed
> allocation for slave 20140916-225614-3125920579-47865-21026-0 in 145691ns
> I0916 22:56:14.062839 21050 master.hpp:861] Adding offer
> 20140916-225614-3125920579-47865-21026-0 with resources cpus(*):2;
> mem(*):1024; disk(*):1024; ports(*):[31000-32000] on slave
> 20140916-225614-3125920579-47865-21026-0 (penates.apache.org)
> I0916 22:56:14.062891 21041 leveldb.cpp:343] Persisting action (16 bytes) to
> leveldb took 110169ns
> I0916 22:56:14.062907 21041 replica.cpp:676] Persisted action at 4
> I0916 22:56:14.062911 21050 master.cpp:3600] Sending 1 offers to framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:14.063065 21043 sched.cpp:544] Scheduler::resourceOffers took
> 39808ns
> I0916 22:56:14.063163 21046 replica.cpp:655] Replica received learned notice
> for position 4
> I0916 22:56:14.063272 21046 leveldb.cpp:343] Persisting action (18 bytes) to
> leveldb took 89981ns
> I0916 22:56:14.063308 21046 leveldb.cpp:401] Deleting ~2 keys from leveldb
> took 18542ns
> I0916 22:56:14.063323 21046 replica.cpp:676] Persisted action at 4
> I0916 22:56:14.063333 21046 replica.cpp:661] Replica learned TRUNCATE action
> at position 4
> I0916 22:56:14.063482 21044 master.hpp:871] Removing offer
> 20140916-225614-3125920579-47865-21026-0 with resources cpus(*):2;
> mem(*):1024; disk(*):1024; ports(*):[31000-32000] on slave
> 20140916-225614-3125920579-47865-21026-0 (penates.apache.org)
> I0916 22:56:14.063535 21044 master.cpp:2201] Processing reply for offers: [
> 20140916-225614-3125920579-47865-21026-0 ] on slave
> 20140916-225614-3125920579-47865-21026-0 at slave(102)@67.195.81.186:47865
> (penates.apache.org) for framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:14.063561 21044 master.cpp:2284] Authorizing framework principal
> 'test-principal' to launch task 1 as user 'jenkins'
> I0916 22:56:14.063824 21040 master.hpp:833] Adding task 1 with resources
> cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] on slave
> 20140916-225614-3125920579-47865-21026-0 (penates.apache.org)
> I0916 22:56:14.063860 21040 master.cpp:2350] Launching task 1 of framework
> 20140916-225614-3125920579-47865-21026-0000 with resources cpus(*):2;
> mem(*):1024; disk(*):1024; ports(*):[31000-32000] on slave
> 20140916-225614-3125920579-47865-21026-0 at slave(102)@67.195.81.186:47865
> (penates.apache.org)
> I0916 22:56:14.063943 21050 slave.cpp:1025] Got assigned task 1 for framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:14.064158 21050 slave.cpp:1135] Launching task 1 for framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:14.065439 21050 slave.cpp:1248] Queuing task '1' for executor 1
> of framework '20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:14.065460 21041 containerizer.cpp:394] Starting container
> 'd383a013-89cf-47c6-ad8e-39e2f3e971fd' for executor '1' of framework
> '20140916-225614-3125920579-47865-21026-0000'
> I0916 22:56:14.065477 21050 slave.cpp:554] Successfully attached file
> '/tmp/HealthCheckTest_HealthStatusChange_cGKTiG/slaves/20140916-225614-3125920579-47865-21026-0/frameworks/20140916-225614-3125920579-47865-21026-0000/executors/1/runs/d383a013-89cf-47c6-ad8e-39e2f3e971fd'
> I0916 22:56:14.066735 21055 launcher.cpp:137] Forked child with pid '21858'
> for container 'd383a013-89cf-47c6-ad8e-39e2f3e971fd'
> I0916 22:56:14.067486 21044 containerizer.cpp:510] Fetching URIs for
> container 'd383a013-89cf-47c6-ad8e-39e2f3e971fd' using command
> '/home/jenkins/jenkins-slave/workspace/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Disable-Java-Disable-Python-Disable-Webui/build/src/mesos-fetcher'
> I0916 22:56:15.037449 21050 hierarchical_allocator_process.hpp:659] Performed
> allocation for 1 slaves in 43708ns
> I0916 22:56:15.038743 21054 slave.cpp:2559] Monitoring executor '1' of
> framework '20140916-225614-3125920579-47865-21026-0000' in container
> 'd383a013-89cf-47c6-ad8e-39e2f3e971fd'
> I0916 22:56:15.078441 21053 slave.cpp:1758] Got registration for executor '1'
> of framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:15.078866 21053 slave.cpp:1876] Flushing queued task 1 for
> executor '1' of framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:15.084800 21043 slave.cpp:2110] Handling status update
> TASK_RUNNING (UUID: a16d2819-e9f4-4119-bde6-f00ad33033e5) for task 1 of
> framework 20140916-225614-3125920579-47865-21026-0000 from
> executor(1)@67.195.81.186:35510
> I0916 22:56:15.084969 21041 status_update_manager.cpp:320] Received status
> update TASK_RUNNING (UUID: a16d2819-e9f4-4119-bde6-f00ad33033e5) for task 1
> of framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:15.084995 21041 status_update_manager.cpp:499] Creating
> StatusUpdate stream for task 1 of framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:15.085160 21041 status_update_manager.cpp:373] Forwarding status
> update TASK_RUNNING (UUID: a16d2819-e9f4-4119-bde6-f00ad33033e5) for task 1
> of framework 20140916-225614-3125920579-47865-21026-0000 to
> [email protected]:47865
> I0916 22:56:15.085314 21043 slave.cpp:2267] Status update manager
> successfully handled status update TASK_RUNNING (UUID:
> a16d2819-e9f4-4119-bde6-f00ad33033e5) for task 1 of framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:15.085332 21041 master.cpp:3212] Forwarding status update
> TASK_RUNNING (UUID: a16d2819-e9f4-4119-bde6-f00ad33033e5) for task 1 of
> framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:15.085335 21043 slave.cpp:2273] Sending acknowledgement for
> status update TASK_RUNNING (UUID: a16d2819-e9f4-4119-bde6-f00ad33033e5) for
> task 1 of framework 20140916-225614-3125920579-47865-21026-0000 to
> executor(1)@67.195.81.186:35510
> I0916 22:56:15.085435 21041 master.cpp:3178] Status update TASK_RUNNING
> (UUID: a16d2819-e9f4-4119-bde6-f00ad33033e5) for task 1 of framework
> 20140916-225614-3125920579-47865-21026-0000 from slave
> 20140916-225614-3125920579-47865-21026-0 at slave(102)@67.195.81.186:47865
> (penates.apache.org)
> I0916 22:56:15.085675 21044 sched.cpp:635] Scheduler::statusUpdate took
> 113998ns
> I0916 22:56:15.085888 21052 master.cpp:2693] Forwarding status update
> acknowledgement a16d2819-e9f4-4119-bde6-f00ad33033e5 for task 1 of framework
> 20140916-225614-3125920579-47865-21026-0000 to slave
> 20140916-225614-3125920579-47865-21026-0 at slave(102)@67.195.81.186:47865
> (penates.apache.org)
> I0916 22:56:15.086109 21051 status_update_manager.cpp:398] Received status
> update acknowledgement (UUID: a16d2819-e9f4-4119-bde6-f00ad33033e5) for task
> 1 of framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:15.086205 21051 slave.cpp:1698] Status update manager
> successfully handled status update acknowledgement (UUID:
> a16d2819-e9f4-4119-bde6-f00ad33033e5) for task 1 of framework
> 20140916-225614-3125920579-47865-21026-0000
> I../../src/tests/health_check_tests.cpp:330: Failure
> Failed to wait 10secs for statusHealth1
> 0916 22:56:16.038705 21049 hierarchical_allocator_process.hpp:659] Performed
> allocation for 1 slaves in 40061ns
> I0916 22:56:16.126260 21052 slave.cpp:2110] Handling status update
> TASK_RUNNING (UUID: 792b8e42-0d72-451b-978a-7d1f29a15751) for task 1 in
> health state healthy of framework 20140916-225614-3125920579-47865-21026-0000
> from executor(1)@67.195.81.186:35510
> I0916 22:56:28.190274 21045 master.cpp:741] Framework
> 20140916-225614-3125920579-47865-21026-0000 disconnected
> I0916 22:56:28.190304 21045 master.cpp:1687] Deactivating framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:19.037235 21050 master.cpp:120] No whitelist given. Advertising
> offers for all slaves
> I0916 22:56:28.190394 21045 master.cpp:763] Giving framework
> 20140916-225614-3125920579-47865-21026-0000 0ns to failover
> ../../src/tests/health_check_tests.cpp:319: Failure
> Actual function call count doesn't match EXPECT_CALL(sched,
> statusUpdate(&driver, _))...
> Expected: to be called 4 times
> Actual: called once - unsatisfied and active
> I0916 22:56:28.190624 21052 slave.cpp:2110] Handling status update
> TASK_RUNNING (UUID: 5783bb6f-112f-4434-a160-a336e890398a) for task 1 in
> health state unhealthy of framework
> 20140916-225614-3125920579-47865-21026-0000 from
> executor(1)@67.195.81.186:35510
> I0916 22:56:28.190757 21052 slave.cpp:2110] Handling status update
> TASK_RUNNING (UUID: b4a9f647-3894-47f3-b55e-49d0355b20f9) for task 1 in
> health state healthy of framework 20140916-225614-3125920579-47865-21026-0000
> from executor(1)@67.195.81.186:35510
> I0916 22:56:28.190773 21046 status_update_manager.cpp:320] Received status
> update TASK_RUNNING (UUID: 792b8e42-0d72-451b-978a-7d1f29a15751) for task 1
> in health state healthy of framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.190831 21040 hierarchical_allocator_process.hpp:405]
> Deactivated framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.190856 21054 master.cpp:3471] Framework failover timeout,
> removing framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.190846 21046 status_update_manager.cpp:373] Forwarding status
> update TASK_RUNNING (UUID: 792b8e42-0d72-451b-978a-7d1f29a15751) for task 1
> in health state healthy of framework
> 20140916-225614-3125920579-47865-21026-0000 to [email protected]:47865
> I0916 22:56:28.190887 21054 master.cpp:3976] Removing framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.190887 21052 slave.cpp:2110] Handling status update
> TASK_RUNNING (UUID: b5d5d6c7-e92c-4ca0-ab72-656542c14ade) for task 1 in
> health state unhealthy of framework
> 20140916-225614-3125920579-47865-21026-0000 from
> executor(1)@67.195.81.186:35510
> I0916 22:56:28.190994 21052 slave.cpp:2110] Handling status update
> TASK_RUNNING (UUID: c0225c5c-b15e-4b5e-a063-07a29703ea12) for task 1 in
> health state healthy of framework 20140916-225614-3125920579-47865-21026-0000
> from executor(1)@67.195.81.186:35510
> I0916 22:56:28.190996 21046 status_update_manager.cpp:320] Received status
> update TASK_RUNNING (UUID: 5783bb6f-112f-4434-a160-a336e890398a) for task 1
> in health state unhealthy of framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.190999 21054 master.hpp:851] Removing task 1 with resources
> cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] on slave
> 20140916-225614-3125920579-47865-21026-0 (penates.apache.org)
> I0916 22:56:28.191090 21046 status_update_manager.cpp:320] Received status
> update TASK_RUNNING (UUID: b4a9f647-3894-47f3-b55e-49d0355b20f9) for task 1
> in health state healthy of framework
> 20140916-225614-3125920579-47865-21026-0000
> W0916 22:56:28.191141 21054 master.cpp:4419] Removing task 1 of framework
> 20140916-225614-3125920579-47865-21026-0000 and slave
> 20140916-225614-3125920579-47865-21026-0 in non-terminal state TASK_RUNNING
> I0916 22:56:28.191093 21052 slave.cpp:2110] Handling status update
> TASK_RUNNING (UUID: e9e3fdb1-d8e0-4bfc-970b-fcd098cace13) for task 1 in
> health state unhealthy of framework
> 20140916-225614-3125920579-47865-21026-0000 from
> executor(1)@67.195.81.186:35510
> I0916 22:56:28.191181 21046 status_update_manager.cpp:320] Received status
> update TASK_RUNNING (UUID: b5d5d6c7-e92c-4ca0-ab72-656542c14ade) for task 1
> in health state unhealthy of framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.191256 21054 master.cpp:650] Master terminating
> I0916 22:56:28.191258 21043 hierarchical_allocator_process.hpp:563] Recovered
> cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] (total
> allocatable: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]) on
> slave 20140916-225614-3125920579-47865-21026-0 from framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.369088 21046 status_update_manager.cpp:320] Received status
> update TASK_RUNNING (UUID: c0225c5c-b15e-4b5e-a063-07a29703ea12) for task 1
> in health state healthy of framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.191319 21052 slave.cpp:2110] Handling status update
> TASK_RUNNING (UUID: 780d211b-6ecc-478d-93e9-6744ed0a2d33) for task 1 in
> health state healthy of framework 20140916-225614-3125920579-47865-21026-0000
> from executor(1)@67.195.81.186:35510
> I0916 22:56:28.369132 21043 hierarchical_allocator_process.hpp:360] Removed
> framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.369225 21046 status_update_manager.cpp:320] Received status
> update TASK_RUNNING (UUID: e9e3fdb1-d8e0-4bfc-970b-fcd098cace13) for task 1
> in health state unhealthy of framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.369283 21052 slave.cpp:2110] Handling status update
> TASK_RUNNING (UUID: f41475b5-9b45-478e-8cd9-2cf7854627dd) for task 1 in
> health state unhealthy of framework
> 20140916-225614-3125920579-47865-21026-0000 from
> executor(1)@67.195.81.186:35510
> I0916 22:56:28.369323 21046 status_update_manager.cpp:320] Received status
> update TASK_RUNNING (UUID: 780d211b-6ecc-478d-93e9-6744ed0a2d33) for task 1
> in health state healthy of framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.369415 21046 status_update_manager.cpp:320] Received status
> update TASK_RUNNING (UUID: f41475b5-9b45-478e-8cd9-2cf7854627dd) for task 1
> in health state unhealthy of framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.369420 21052 slave.cpp:2110] Handling status update
> TASK_RUNNING (UUID: 5f3a1b44-51f0-4deb-ba4c-e7238f63f856) for task 1 in
> health state healthy of framework 20140916-225614-3125920579-47865-21026-0000
> from executor(1)@67.195.81.186:35510
> I0916 22:56:28.369536 21052 slave.cpp:2110] Handling status update
> TASK_RUNNING (UUID: 4663a09b-147f-455c-a577-3d967ddf5256) for task 1 in
> health state unhealthy of framework
> 20140916-225614-3125920579-47865-21026-0000 from
> executor(1)@67.195.81.186:35510
> I0916 22:56:28.369642 21052 slave.cpp:2110] Handling status update
> TASK_RUNNING (UUID: 89cbabd7-0169-4b58-8df7-d8fd4bc4a287) for task 1 in
> health state healthy of framework 20140916-225614-3125920579-47865-21026-0000
> from executor(1)@67.195.81.186:35510
> I0916 22:56:28.369685 21055 status_update_manager.cpp:320] Received status
> update TASK_RUNNING (UUID: 5f3a1b44-51f0-4deb-ba4c-e7238f63f856) for task 1
> in health state healthy of framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.369753 21052 slave.cpp:2110] Handling status update
> TASK_RUNNING (UUID: 3c491f72-95f1-4c52-b7ca-c6470f748eb5) for task 1 in
> health state unhealthy of framework
> 20140916-225614-3125920579-47865-21026-0000 from
> executor(1)@67.195.81.186:35510
> I0916 22:56:28.369802 21055 status_update_manager.cpp:320] Received status
> update TASK_RUNNING (UUID: 4663a09b-147f-455c-a577-3d967ddf5256) for task 1
> in health state unhealthy of framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.369884 21055 status_update_manager.cpp:320] Received status
> update TASK_RUNNING (UUID: 89cbabd7-0169-4b58-8df7-d8fd4bc4a287) for task 1
> in health state healthy of framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.369889 21052 slave.cpp:2110] Handling status update
> TASK_RUNNING (UUID: 218be9bd-a229-4808-8fb6-1e507830cdaf) for task 1 in
> health state healthy of framework 20140916-225614-3125920579-47865-21026-0000
> from executor(1)@67.195.81.186:35510
> I0916 22:56:28.369943 21055 status_update_manager.cpp:320] Received status
> update TASK_RUNNING (UUID: 3c491f72-95f1-4c52-b7ca-c6470f748eb5) for task 1
> in health state unhealthy of framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.369978 21052 slave.cpp:1431] Asked to shut down framework
> 20140916-225614-3125920579-47865-21026-0000 by [email protected]:47865
> I0916 22:56:28.369998 21052 slave.cpp:1456] Shutting down framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.370009 21055 status_update_manager.cpp:320] Received status
> update TASK_RUNNING (UUID: 218be9bd-a229-4808-8fb6-1e507830cdaf) for task 1
> in health state healthy of framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.370018 21052 slave.cpp:2899] Shutting down executor '1' of
> framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.370183 21052 slave.cpp:2267] Status update manager
> successfully handled status update TASK_RUNNING (UUID:
> 792b8e42-0d72-451b-978a-7d1f29a15751) for task 1 in health state healthy of
> framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.370206 21052 slave.cpp:2273] Sending acknowledgement for
> status update TASK_RUNNING (UUID: 792b8e42-0d72-451b-978a-7d1f29a15751) for
> task 1 in health state healthy of framework
> 20140916-225614-3125920579-47865-21026-0000 to executor(1)@67.195.81.186:35510
> I0916 22:56:28.370426 21052 slave.cpp:2267] Status update manager
> successfully handled status update TASK_RUNNING (UUID:
> 5783bb6f-112f-4434-a160-a336e890398a) for task 1 in health state unhealthy of
> framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.370447 21052 slave.cpp:2273] Sending acknowledgement for
> status update TASK_RUNNING (UUID: 5783bb6f-112f-4434-a160-a336e890398a) for
> task 1 in health state unhealthy of framework
> 20140916-225614-3125920579-47865-21026-0000 to executor(1)@67.195.81.186:35510
> I0916 22:56:28.370635 21052 slave.cpp:2267] Status update manager
> successfully handled status update TASK_RUNNING (UUID:
> b4a9f647-3894-47f3-b55e-49d0355b20f9) for task 1 in health state healthy of
> framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.370657 21052 slave.cpp:2273] Sending acknowledgement for
> status update TASK_RUNNING (UUID: b4a9f647-3894-47f3-b55e-49d0355b20f9) for
> task 1 in health state healthy of framework
> 20140916-225614-3125920579-47865-21026-0000 to executor(1)@67.195.81.186:35510
> I0916 22:56:28.370815 21052 slave.cpp:2267] Status update manager
> successfully handled status update TASK_RUNNING (UUID:
> b5d5d6c7-e92c-4ca0-ab72-656542c14ade) for task 1 in health state unhealthy of
> framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.370837 21052 slave.cpp:2273] Sending acknowledgement for
> status update TASK_RUNNING (UUID: b5d5d6c7-e92c-4ca0-ab72-656542c14ade) for
> task 1 in health state unhealthy of framework
> 20140916-225614-3125920579-47865-21026-0000 to executor(1)@67.195.81.186:35510
> I0916 22:56:28.370972 21052 slave.cpp:2267] Status update manager
> successfully handled status update TASK_RUNNING (UUID:
> c0225c5c-b15e-4b5e-a063-07a29703ea12) for task 1 in health state healthy of
> framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.371000 21052 slave.cpp:2273] Sending acknowledgement for
> status update TASK_RUNNING (UUID: c0225c5c-b15e-4b5e-a063-07a29703ea12) for
> task 1 in health state healthy of framework
> 20140916-225614-3125920579-47865-21026-0000 to executor(1)@67.195.81.186:35510
> I0916 22:56:28.371155 21052 slave.cpp:2378] [email protected]:47865 exited
> W0916 22:56:28.371177 21052 slave.cpp:2381] Master disconnected! Waiting for
> a new master to be elected
> I0916 22:56:28.371202 21052 slave.cpp:2267] Status update manager
> successfully handled status update TASK_RUNNING (UUID:
> e9e3fdb1-d8e0-4bfc-970b-fcd098cace13) for task 1 in health state unhealthy of
> framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.540035 21052 slave.cpp:2273] Sending acknowledgement for
> status update TASK_RUNNING (UUID: e9e3fdb1-d8e0-4bfc-970b-fcd098cace13) for
> task 1 in health state unhealthy of framework
> 20140916-225614-3125920579-47865-21026-0000 to executor(1)@67.195.81.186:35510
> I0916 22:56:28.371701 21053 containerizer.cpp:882] Destroying container
> 'd383a013-89cf-47c6-ad8e-39e2f3e971fd'
> I0916 22:56:28.540177 21052 slave.cpp:2267] Status update manager
> successfully handled status update TASK_RUNNING (UUID:
> 780d211b-6ecc-478d-93e9-6744ed0a2d33) for task 1 in health state healthy of
> framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.540196 21052 slave.cpp:2273] Sending acknowledgement for
> status update TASK_RUNNING (UUID: 780d211b-6ecc-478d-93e9-6744ed0a2d33) for
> task 1 in health state healthy of framework
> 20140916-225614-3125920579-47865-21026-0000 to executor(1)@67.195.81.186:35510
> I0916 22:56:28.540324 21052 slave.cpp:2267] Status update manager
> successfully handled status update TASK_RUNNING (UUID:
> f41475b5-9b45-478e-8cd9-2cf7854627dd) for task 1 in health state unhealthy of
> framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.540350 21052 slave.cpp:2273] Sending acknowledgement for
> status update TASK_RUNNING (UUID: f41475b5-9b45-478e-8cd9-2cf7854627dd) for
> task 1 in health state unhealthy of framework
> 20140916-225614-3125920579-47865-21026-0000 to executor(1)@67.195.81.186:35510
> I0916 22:56:28.540403 21052 slave.cpp:2267] Status update manager
> successfully handled status update TASK_RUNNING (UUID:
> 5f3a1b44-51f0-4deb-ba4c-e7238f63f856) for task 1 in health state healthy of
> framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.540421 21052 slave.cpp:2273] Sending acknowledgement for
> status update TASK_RUNNING (UUID: 5f3a1b44-51f0-4deb-ba4c-e7238f63f856) for
> task 1 in health state healthy of framework
> 20140916-225614-3125920579-47865-21026-0000 to executor(1)@67.195.81.186:35510
> I0916 22:56:28.540530 21052 slave.cpp:2267] Status update manager
> successfully handled status update TASK_RUNNING (UUID:
> 4663a09b-147f-455c-a577-3d967ddf5256) for task 1 in health state unhealthy of
> framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.540556 21052 slave.cpp:2273] Sending acknowledgement for
> status update TASK_RUNNING (UUID: 4663a09b-147f-455c-a577-3d967ddf5256) for
> task 1 in health state unhealthy of framework
> 20140916-225614-3125920579-47865-21026-0000 to executor(1)@67.195.81.186:35510
> I0916 22:56:28.540664 21052 slave.cpp:2267] Status update manager
> successfully handled status update TASK_RUNNING (UUID:
> 89cbabd7-0169-4b58-8df7-d8fd4bc4a287) for task 1 in health state healthy of
> framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.540681 21052 slave.cpp:2273] Sending acknowledgement for
> status update TASK_RUNNING (UUID: 89cbabd7-0169-4b58-8df7-d8fd4bc4a287) for
> task 1 in health state healthy of framework
> 20140916-225614-3125920579-47865-21026-0000 to executor(1)@67.195.81.186:35510
> I0916 22:56:28.540889 21052 slave.cpp:2267] Status update manager
> successfully handled status update TASK_RUNNING (UUID:
> 3c491f72-95f1-4c52-b7ca-c6470f748eb5) for task 1 in health state unhealthy of
> framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.540918 21052 slave.cpp:2273] Sending acknowledgement for
> status update TASK_RUNNING (UUID: 3c491f72-95f1-4c52-b7ca-c6470f748eb5) for
> task 1 in health state unhealthy of framework
> 20140916-225614-3125920579-47865-21026-0000 to executor(1)@67.195.81.186:35510
> I0916 22:56:28.541082 21052 slave.cpp:2267] Status update manager
> successfully handled status update TASK_RUNNING (UUID:
> 218be9bd-a229-4808-8fb6-1e507830cdaf) for task 1 in health state healthy of
> framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:28.541111 21052 slave.cpp:2273] Sending acknowledgement for
> status update TASK_RUNNING (UUID: 218be9bd-a229-4808-8fb6-1e507830cdaf) for
> task 1 in health state healthy of framework
> 20140916-225614-3125920579-47865-21026-0000 to executor(1)@67.195.81.186:35510
> I0916 22:56:29.047708 21053 containerizer.cpp:997] Executor for container
> 'd383a013-89cf-47c6-ad8e-39e2f3e971fd' has exited
> I0916 22:56:29.048037 21050 slave.cpp:2617] Executor '1' of framework
> 20140916-225614-3125920579-47865-21026-0000 terminated with signal Killed
> I0916 22:56:29.048197 21050 slave.cpp:2753] Cleaning up executor '1' of
> framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:29.048373 21050 slave.cpp:2828] Cleaning up framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:29.048444 21043 status_update_manager.cpp:282] Closing status
> update streams for framework 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:29.048457 21050 slave.cpp:477] Slave terminating
> I0916 22:56:29.048476 21043 status_update_manager.cpp:530] Cleaning up status
> update stream for task 1 of framework
> 20140916-225614-3125920579-47865-21026-0000
> I0916 22:56:29.048462 21041 gc.cpp:56] Scheduling
> '/tmp/HealthCheckTest_HealthStatusChange_cGKTiG/slaves/20140916-225614-3125920579-47865-21026-0/frameworks/20140916-225614-3125920579-47865-21026-0000/executors/1/runs/d383a013-89cf-47c6-ad8e-39e2f3e971fd'
> for gc 6.99999944121481days in the future
> I0916 22:56:29.048568 21041 gc.cpp:56] Scheduling
> '/tmp/HealthCheckTest_HealthStatusChange_cGKTiG/slaves/20140916-225614-3125920579-47865-21026-0/frameworks/20140916-225614-3125920579-47865-21026-0000/executors/1'
> for gc 6.99999944031111days in the future
> I0916 22:56:29.048607 21041 gc.cpp:56] Scheduling
> '/tmp/HealthCheckTest_HealthStatusChange_cGKTiG/slaves/20140916-225614-3125920579-47865-21026-0/frameworks/20140916-225614-3125920579-47865-21026-0000'
> for gc 6.99999943939852days in the future
> [ FAILED ] HealthCheckTest.HealthStatusChange (15019 ms)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
