[
https://issues.apache.org/jira/browse/MESOS-8305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16532716#comment-16532716
]
Till Toenshoff commented on MESOS-8305:
---------------------------------------
Just seen this test being flaky on 1.4.x. The errors however are not identical
to what is described above.
{noformat}
08:36:56 [ RUN ]
ROOT_DOCKER_DockerAndMesosContainerizers/DefaultExecutorTest.ROOT_MultiTaskgroupSharePidNamespace/0
08:36:56 I0702 08:36:56.885752 28719 cluster.cpp:162] Creating default 'local'
authorizer
08:36:56 I0702 08:36:56.886807 18346 master.cpp:442] Master
1b41a301-e70a-4aec-830f-7850032b28c3 (ip-172-16-10-112.ec2.internal) started on
172.16.10.112:35906
08:36:56 I0702 08:36:56.886837 18346 master.cpp:444] Flags at startup:
--acls="" --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins"
--allocation_interval="1secs" --allocator="hierarchical"
--authenticate_agents="true" --authenticate_frameworks="true"
--authenticate_http_frameworks="true" --authenticate_http_readonly="true"
--authenticate_http_readwrite="true" --authenticators="crammd5"
--authorizers="local" --credentials="/tmp/f7z9P0/credentials"
--filter_gpu_resources="true" --framework_sorter="drf" --help="false"
--hostname_lookup="true" --http_authenticators="basic"
--http_framework_authenticators="basic" --initialize_driver_logging="true"
--log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
--max_agent_ping_timeouts="5" --max_completed_frameworks="50"
--max_completed_tasks_per_framework="1000"
--max_unreachable_tasks_per_framework="1000" --port="5050" --quiet="false"
--recovery_agent_removal_limit="100%" --registry="in_memory"
--registry_fetch_timeout="1mins" --registry_gc_interval="15mins"
--registry_max_agent_age="2weeks" --registry_max_agent_count="102400"
--registry_store_timeout="100secs" --registry_strict="false"
--role_sorter="drf" --root_submissions="true" --version="false"
--webui_dir="/usr/local/share/mesos/webui" --work_dir="/tmp/f7z9P0/master"
--zk_session_timeout="10secs"
08:36:56 I0702 08:36:56.886965 18346 master.cpp:494] Master only allowing
authenticated frameworks to register
08:36:56 I0702 08:36:56.886974 18346 master.cpp:508] Master only allowing
authenticated agents to register
08:36:56 I0702 08:36:56.886981 18346 master.cpp:521] Master only allowing
authenticated HTTP frameworks to register
08:36:56 I0702 08:36:56.886986 18346 credentials.hpp:37] Loading credentials
for authentication from '/tmp/f7z9P0/credentials'
08:36:56 I0702 08:36:56.887074 18346 master.cpp:566] Using default 'crammd5'
authenticator
08:36:56 I0702 08:36:56.887127 18346 http.cpp:1045] Creating default 'basic'
HTTP authenticator for realm 'mesos-master-readonly'
08:36:56 I0702 08:36:56.887168 18346 http.cpp:1045] Creating default 'basic'
HTTP authenticator for realm 'mesos-master-readwrite'
08:36:56 I0702 08:36:56.887195 18346 http.cpp:1045] Creating default 'basic'
HTTP authenticator for realm 'mesos-master-scheduler'
08:36:56 I0702 08:36:56.887310 18346 master.cpp:646] Authorization enabled
08:36:56 I0702 08:36:56.887361 18348 hierarchical.cpp:171] Initialized
hierarchical allocator process
08:36:56 I0702 08:36:56.887409 18348 whitelist_watcher.cpp:77] No whitelist
given
08:36:56 I0702 08:36:56.888077 18349 master.cpp:2163] Elected as the leading
master!
08:36:56 I0702 08:36:56.888098 18349 master.cpp:1702] Recovering from registrar
08:36:56 I0702 08:36:56.888209 18349 registrar.cpp:347] Recovering registrar
08:36:56 I0702 08:36:56.888391 18349 registrar.cpp:391] Successfully fetched
the registry (0B) in 159232ns
08:36:56 I0702 08:36:56.888428 18349 registrar.cpp:495] Applied 1 operations in
13259ns; attempting to update the registry
08:36:56 I0702 08:36:56.888618 18347 registrar.cpp:552] Successfully updated
the registry in 174080ns
08:36:56 I0702 08:36:56.888662 18347 registrar.cpp:424] Successfully recovered
registrar
08:36:56 I0702 08:36:56.888756 18347 master.cpp:1801] Recovered 0 agents from
the registry (172B); allowing 10mins for agents to re-register
08:36:56 I0702 08:36:56.888878 18349 hierarchical.cpp:209] Skipping recovery of
hierarchical allocator: nothing to recover
08:36:56 W0702 08:36:56.890381 28719 process.cpp:3196] Attempted to spawn
already running process [email protected]:35906
08:36:56 I0702 08:36:56.978411 28719 containerizer.cpp:246] Using isolation:
cgroups/cpu,filesystem/linux,namespaces/pid,network/cni,volume/image,environment_secret
08:36:56 I0702 08:36:56.982765 28719 linux_launcher.cpp:150] Using
/sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
08:36:56 I0702 08:36:56.983234 28719 provisioner.cpp:255] Using default backend
'overlay'
08:36:56 I0702 08:36:56.990206 28719 cluster.cpp:448] Creating default 'local'
authorizer
08:36:56 I0702 08:36:56.990712 18348 slave.cpp:250] Mesos agent started on
(820)@172.16.10.112:35906
08:36:56 I0702 08:36:56.990886 18348 slave.cpp:251] Flags at startup: --acls=""
--appc_simple_discovery_uri_prefix="http://";
--appc_store_dir="/tmp/ROOT_DOCKER_DockerAndMesosContainerizers_DefaultExecutorTest_ROOT_MultiTaskgroupSharePidNamespace_0_U0uIz8/store/appc"
--authenticate_http_readonly="true" --authenticate_http_readwrite="false"
--authenticatee="crammd5" --authentication_backoff_factor="1secs"
--authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false"
--cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup"
--cgroups_limit_swap="false" --cgroups_root="mesos"
--container_disk_watch_interval="15secs" --containerizers="docker,mesos"
--credential="/tmp/ROOT_DOCKER_DockerAndMesosContainerizers_DefaultExecutorTest_ROOT_MultiTaskgroupSharePidNamespace_0_U0uIz8/credential"
--default_role="*" --disallow_sharing_agent_pid_namespace="false"
--disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true"
--docker_registry="https://registry-1.docker.io"; --docker_remove_delay="6hrs"
--docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns"
--docker_store_dir="/tmp/ROOT_DOCKER_DockerAndMesosContainerizers_DefaultExecutorTest_ROOT_MultiTaskgroupSharePidNamespace_0_U0uIz8/store/docker"
--docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume"
--enforce_container_disk_quota="false" --executor_registration_timeout="1mins"
--executor_reregistration_timeout="2secs"
--executor_shutdown_grace_period="5secs"
--fetcher_cache_dir="/tmp/ROOT_DOCKER_DockerAndMesosContainerizers_DefaultExecutorTest_ROOT_MultiTaskgroupSharePidNamespace_0_U0uIz8/fetch"
--fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks"
--gc_disk_headroom="0.1" --hadoop_home="" --help="false"
--hostname_lookup="true" --http_command_executor="false"
--http_credentials="/tmp/ROOT_DOCKER_DockerAndMesosContainerizers_DefaultExecutorTest_ROOT_MultiTaskgroupSharePidNamespace_0_U0uIz8/http_credentials"
--http_heartbeat_interval="30secs" --initialize_driver_logging="true"
--isolation="cgroups/cpu,filesystem/linux,namespaces/pid" --launcher="linux"
--launcher_dir="/home/ubuntu/workspace/mesos/Mesos_CI-build/FLAG/Clang/label/mesos-ec2-ubuntu-16.04/mesos/build/src"
--logbufsecs="0" --logging_level="INFO"
--max_completed_executors_per_framework="150"
--oversubscribed_resources_interval="15secs" --perf_duration="10secs"
--perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns"
--quiet="false" --recover="reconnect" --recovery_timeout="15mins"
--registration_backoff_factor="10ms"
--resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]"
--revocable_cpu_low_priority="true"
--runtime_dir="/tmp/ROOT_DOCKER_DockerAndMesosContainerizers_DefaultExecutorTest_ROOT_MultiTaskgroupSharePidNamespace_0_U0uIz8"
--sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true"
--systemd_enable_support="true"
--systemd_runtime_directory="/run/systemd/system" --version="false"
--work_dir="/tmp/ROOT_DOCKER_DockerAndMesosContainerizers_DefaultExecutorTest_ROOT_MultiTaskgroupSharePidNamespace_0_0fWVH4"
08:36:56 I0702 08:36:56.991469 18348 credentials.hpp:86] Loading credential for
authentication from
'/tmp/ROOT_DOCKER_DockerAndMesosContainerizers_DefaultExecutorTest_ROOT_MultiTaskgroupSharePidNamespace_0_U0uIz8/credential'
08:36:56 I0702 08:36:56.991209 28719 scheduler.cpp:184] Version: 1.4.2
08:36:56 I0702 08:36:56.991696 18348 slave.cpp:283] Agent using credential for:
test-principal
08:36:56 I0702 08:36:56.991720 18348 credentials.hpp:37] Loading credentials
for authentication from
'/tmp/ROOT_DOCKER_DockerAndMesosContainerizers_DefaultExecutorTest_ROOT_MultiTaskgroupSharePidNamespace_0_U0uIz8/http_credentials'
08:36:56 I0702 08:36:56.991875 18348 http.cpp:1045] Creating default 'basic'
HTTP authenticator for realm 'mesos-agent-readonly'
08:36:56 I0702 08:36:56.992400 18349 scheduler.cpp:470] New master detected at
[email protected]:35906
08:36:56 I0702 08:36:56.992429 18349 scheduler.cpp:479] Waiting for 0ns before
initiating a re-(connection) attempt with the master
08:36:56 I0702 08:36:56.992621 18348 slave.cpp:565] Agent resources:
[{"name":"cpus","scalar":{"value":2.0},"type":"SCALAR"},{"name":"mem","scalar":{"value":1024.0},"type":"SCALAR"},{"name":"disk","scalar":{"value":1024.0},"type":"SCALAR"},{"name":"ports","ranges":{"range":[{"begin":31000,"end":32000}]},"type":"RANGES"}]
08:36:56 I0702 08:36:56.992702 18348 slave.cpp:573] Agent attributes: [ ]
08:36:56 I0702 08:36:56.992710 18348 slave.cpp:582] Agent hostname:
ip-172-16-10-112.ec2.internal
08:36:56 I0702 08:36:56.992854 18350 status_update_manager.cpp:177] Pausing
sending status updates
08:36:56 I0702 08:36:56.993185 18348 state.cpp:64] Recovering state from
'/tmp/ROOT_DOCKER_DockerAndMesosContainerizers_DefaultExecutorTest_ROOT_MultiTaskgroupSharePidNamespace_0_0fWVH4/meta'
08:36:56 I0702 08:36:56.993363 18348 status_update_manager.cpp:203] Recovering
status update manager
08:36:56 I0702 08:36:56.993469 18348 containerizer.cpp:609] Recovering
containerizer
08:36:56 I0702 08:36:56.993595 18347 docker.cpp:890] Recovering Docker
containers
08:36:56 I0702 08:36:56.993783 18347 docker.cpp:1397] Running docker -H
unix:///var/run/docker.sock ps -a
08:36:57 I0702 08:36:56.999671 18351 scheduler.cpp:361] Connected with the
master at http://172.16.10.112:35906/master/api/v1/scheduler
08:36:57 I0702 08:36:57.000233 18350 scheduler.cpp:243] Sending SUBSCRIBE call
to http://172.16.10.112:35906/master/api/v1/scheduler
08:36:57 I0702 08:36:57.002498 18348 provisioner.cpp:416] Provisioner recovery
complete
08:36:57 I0702 08:36:57.003031 18352 process.cpp:3938] Handling HTTP event for
process 'master' with path: '/master/api/v1/scheduler'
08:36:57 I0702 08:36:57.003875 18352 http.cpp:1185] HTTP POST for
/master/api/v1/scheduler from 172.16.10.112:38732
08:36:57 I0702 08:36:57.004159 18352 master.cpp:2552] Received subscription
request for HTTP framework 'default'
08:36:57 I0702 08:36:57.004354 18352 master.cpp:2235] Authorizing framework
principal 'test-principal' to receive offers for roles '{ * }'
08:36:57 I0702 08:36:57.004715 18352 master.cpp:2688] Subscribing framework
'default' with checkpointing disabled and capabilities [ MULTI_ROLE,
RESERVATION_REFINEMENT ]
08:36:57 I0702 08:36:57.005127 18352 hierarchical.cpp:295] Added framework
1b41a301-e70a-4aec-830f-7850032b28c3-0000
08:36:57 I0702 08:36:57.005954 18352 hierarchical.cpp:2100] No allocations
performed
08:36:57 I0702 08:36:57.006136 18352 hierarchical.cpp:2190] No inverse offers
to send out!
08:36:57 I0702 08:36:57.006312 18352 hierarchical.cpp:1438] Performed
allocation for 0 agents in 379164ns
08:36:57 I0702 08:36:57.005762 18351 scheduler.cpp:676] Enqueuing event
SUBSCRIBED received from http://172.16.10.112:35906/master/api/v1/scheduler
08:36:57
08:36:57 I0702 08:36:57.005344 18346 master.hpp:2217] Sending heartbeat to
1b41a301-e70a-4aec-830f-7850032b28c3-0000
08:36:57 GMOCK WARNING:
08:36:57 Uninteresting mock function call - returning directly.
08:36:57 Function call: subscribed(0x7ffcfade4888, @0x7f49c000b980 48-byte
object <40-EB 8F-ED 49-7F 00-00 00-00 00-00 00-00 00-00 07-00 00-00 00-00 00-00
40-0F 02-C0 49-7F 00-00 80-72 03-C0 49-7F 00-00 00-00 00-00 00-00 2E-40>)
08:36:57 NOTE: You can safely ignore the above warning unless this call should
not happen. Do not suppress it by blindly adding an EXPECT_CALL() if you don't
mean to enforce the call. See
https://github.com/google/googletest/blob/master/googlemock/docs/CookBook.md#knowing-when-to-expect
for details.
08:36:57 I0702 08:36:57.007515 18351 scheduler.cpp:676] Enqueuing event
HEARTBEAT received from http://172.16.10.112:35906/master/api/v1/scheduler
08:36:57
08:36:57 GMOCK WARNING:
08:36:57 Uninteresting mock function call - returning directly.
08:36:57 Function call: heartbeat(0x7ffcfade4888)
08:36:57 NOTE: You can safely ignore the above warning unless this call should
not happen. Do not suppress it by blindly adding an EXPECT_CALL() if you don't
mean to enforce the call. See
https://github.com/google/googletest/blob/master/googlemock/docs/CookBook.md#knowing-when-to-expect
for details.
08:36:57 I0702 08:36:57.079345 18348 slave.cpp:6325] Finished recovery
08:36:57 I0702 08:36:57.079663 18348 slave.cpp:6507] Querying resource
estimator for oversubscribable resources
08:36:57 I0702 08:36:57.079736 18348 slave.cpp:6521] Received oversubscribable
resources {} from the resource estimator
08:36:57 I0702 08:36:57.079915 18350 status_update_manager.cpp:177] Pausing
sending status updates
08:36:57 I0702 08:36:57.079912 18349 slave.cpp:971] New master detected at
[email protected]:35906
08:36:57 I0702 08:36:57.080047 18349 slave.cpp:1006] Detecting new master
08:36:57 I0702 08:36:57.081328 18351 slave.cpp:1033] Authenticating with master
[email protected]:35906
08:36:57 I0702 08:36:57.081519 18351 slave.cpp:1044] Using default CRAM-MD5
authenticatee
08:36:57 I0702 08:36:57.081629 18351 authenticatee.cpp:121] Creating new client
SASL connection
08:36:57 I0702 08:36:57.082093 18351 master.cpp:7867] Authenticating
slave(820)@172.16.10.112:35906
08:36:57 I0702 08:36:57.082156 18351 authenticator.cpp:414] Starting
authentication session for crammd5-authenticatee(1527)@172.16.10.112:35906
08:36:57 I0702 08:36:57.082257 18351 authenticator.cpp:98] Creating new server
SASL connection
08:36:57 I0702 08:36:57.082566 18351 authenticatee.cpp:213] Received SASL
authentication mechanisms: CRAM-MD5
08:36:57 I0702 08:36:57.082587 18351 authenticatee.cpp:239] Attempting to
authenticate with mechanism 'CRAM-MD5'
08:36:57 I0702 08:36:57.082617 18351 authenticator.cpp:204] Received SASL
authentication start
08:36:57 I0702 08:36:57.082653 18351 authenticator.cpp:326] Authentication
requires more steps
08:36:57 I0702 08:36:57.082720 18351 authenticatee.cpp:259] Received SASL
authentication step
08:36:57 I0702 08:36:57.082895 18351 authenticator.cpp:232] Received SASL
authentication step
08:36:57 I0702 08:36:57.082924 18351 auxprop.cpp:109] Request to lookup
properties for user: 'test-principal' realm: 'ip-172-16-10-112.ec2.internal'
server FQDN: 'ip-172-16-10-112.ec2.internal' SASL_AUXPROP_VERIFY_AGAINST_HASH:
false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false
08:36:57 I0702 08:36:57.082934 18351 auxprop.cpp:181] Looking up auxiliary
property '*userPassword'
08:36:57 I0702 08:36:57.082947 18351 auxprop.cpp:181] Looking up auxiliary
property '*cmusaslsecretCRAM-MD5'
08:36:57 I0702 08:36:57.082957 18351 auxprop.cpp:109] Request to lookup
properties for user: 'test-principal' realm: 'ip-172-16-10-112.ec2.internal'
server FQDN: 'ip-172-16-10-112.ec2.internal' SASL_AUXPROP_VERIFY_AGAINST_HASH:
false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true
08:36:57 I0702 08:36:57.082964 18351 auxprop.cpp:131] Skipping auxiliary
property '*userPassword' since SASL_AUXPROP_AUTHZID == true
08:36:57 I0702 08:36:57.082970 18351 auxprop.cpp:131] Skipping auxiliary
property '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true
08:36:57 I0702 08:36:57.082983 18351 authenticator.cpp:318] Authentication
success
08:36:57 I0702 08:36:57.083117 18352 master.cpp:7897] Successfully
authenticated principal 'test-principal' at slave(820)@172.16.10.112:35906
08:36:57 I0702 08:36:57.083082 18351 authenticatee.cpp:299] Authentication
success
08:36:57 I0702 08:36:57.083359 18351 slave.cpp:1128] Successfully authenticated
with master [email protected]:35906
08:36:57 I0702 08:36:57.083448 18351 slave.cpp:1600] Will retry registration in
8.852079ms if necessary
08:36:57 I0702 08:36:57.083689 18347 master.cpp:5721] Received register agent
message from slave(820)@172.16.10.112:35906 (ip-172-16-10-112.ec2.internal)
08:36:57 I0702 08:36:57.083724 18347 master.cpp:3810] Authorizing agent with
principal 'test-principal'
08:36:57 I0702 08:36:57.083247 18346 authenticator.cpp:432] Authentication
session cleanup for crammd5-authenticatee(1527)@172.16.10.112:35906
08:36:57 I0702 08:36:57.083999 18347 master.cpp:5781] Authorized registration
of agent at slave(820)@172.16.10.112:35906 (ip-172-16-10-112.ec2.internal)
08:36:57 I0702 08:36:57.084048 18347 master.cpp:5874] Registering agent at
slave(820)@172.16.10.112:35906 (ip-172-16-10-112.ec2.internal) with id
1b41a301-e70a-4aec-830f-7850032b28c3-S0
08:36:57 I0702 08:36:57.084358 18347 registrar.cpp:495] Applied 1 operations in
23525ns; attempting to update the registry
08:36:57 I0702 08:36:57.084637 18347 registrar.cpp:552] Successfully updated
the registry in 244992ns
08:36:57 I0702 08:36:57.084738 18347 master.cpp:5921] Admitted agent
1b41a301-e70a-4aec-830f-7850032b28c3-S0 at slave(820)@172.16.10.112:35906
(ip-172-16-10-112.ec2.internal)
08:36:57 I0702 08:36:57.084888 18350 slave.cpp:5000] Received ping from
slave-observer(748)@172.16.10.112:35906
08:36:57 I0702 08:36:57.085072 18352 hierarchical.cpp:570] Added agent
1b41a301-e70a-4aec-830f-7850032b28c3-S0 (ip-172-16-10-112.ec2.internal) with
cpus:2; mem:1024; disk:1024; ports:[31000-32000] (allocated: {})
08:36:57 I0702 08:36:57.084885 18347 master.cpp:5952] Registered agent
1b41a301-e70a-4aec-830f-7850032b28c3-S0 at slave(820)@172.16.10.112:35906
(ip-172-16-10-112.ec2.internal) with
[{"name":"cpus","scalar":{"value":2.0},"type":"SCALAR"},{"name":"mem","scalar":{"value":1024.0},"type":"SCALAR"},{"name":"disk","scalar":{"value":1024.0},"type":"SCALAR"},{"name":"ports","ranges":{"range":[{"begin":31000,"end":32000}]},"type":"RANGES"}]
08:36:57 I0702 08:36:57.085233 18350 slave.cpp:1174] Registered with master
[email protected]:35906; given agent ID
1b41a301-e70a-4aec-830f-7850032b28c3-S0
08:36:57 I0702 08:36:57.085477 18352 hierarchical.cpp:2190] No inverse offers
to send out!
08:36:57 I0702 08:36:57.085506 18352 hierarchical.cpp:1438] Performed
allocation for 1 agents in 236577ns
08:36:57 I0702 08:36:57.085530 18352 status_update_manager.cpp:184] Resuming
sending status updates
08:36:57 I0702 08:36:57.085664 18352 master.cpp:7697] Sending 1 offers to
framework 1b41a301-e70a-4aec-830f-7850032b28c3-0000 (default)
08:36:57 I0702 08:36:57.085849 18350 slave.cpp:1194] Checkpointing SlaveInfo to
'/tmp/ROOT_DOCKER_DockerAndMesosContainerizers_DefaultExecutorTest_ROOT_MultiTaskgroupSharePidNamespace_0_0fWVH4/meta/slaves/1b41a301-e70a-4aec-830f-7850032b28c3-S0/slave.info'
08:36:57 I0702 08:36:57.086319 18352 scheduler.cpp:676] Enqueuing event OFFERS
received from http://172.16.10.112:35906/master/api/v1/scheduler
08:36:57 I0702 08:36:57.086596 18350 slave.cpp:1243] Forwarding total
oversubscribed resources {}
08:36:57 I0702 08:36:57.086660 18350 master.cpp:6701] Received update of agent
1b41a301-e70a-4aec-830f-7850032b28c3-S0 at slave(820)@172.16.10.112:35906
(ip-172-16-10-112.ec2.internal) with total oversubscribed resources {}
08:36:57 I0702 08:36:57.086768 18350 hierarchical.cpp:639] Agent
1b41a301-e70a-4aec-830f-7850032b28c3-S0 (ip-172-16-10-112.ec2.internal) updated
with total resources cpus:2; mem:1024; disk:1024; ports:[31000-32000]
08:36:57 I0702 08:36:57.888594 18348 hierarchical.cpp:2100] No allocations
performed
08:36:57 I0702 08:36:57.888639 18348 hierarchical.cpp:2190] No inverse offers
to send out!
08:36:57 I0702 08:36:57.888651 18348 hierarchical.cpp:1438] Performed
allocation for 1 agents in 150902ns
08:36:58 I0702 08:36:58.889405 18350 hierarchical.cpp:2100] No allocations
performed
08:36:58 I0702 08:36:58.889448 18350 hierarchical.cpp:2190] No inverse offers
to send out!
08:36:58 I0702 08:36:58.889461 18350 hierarchical.cpp:1438] Performed
allocation for 1 agents in 149977ns
08:36:59 I0702 08:36:59.890436 18346 hierarchical.cpp:2100] No allocations
performed
08:36:59 I0702 08:36:59.890481 18346 hierarchical.cpp:2190] No inverse offers
to send out!
08:36:59 I0702 08:36:59.890494 18346 hierarchical.cpp:1438] Performed
allocation for 1 agents in 145043ns
08:37:00 I0702 08:37:00.891070 18345 hierarchical.cpp:2100] No allocations
performed
08:37:00 I0702 08:37:00.891116 18345 hierarchical.cpp:2190] No inverse offers
to send out!
08:37:00 I0702 08:37:00.891129 18345 hierarchical.cpp:1438] Performed
allocation for 1 agents in 153807ns
08:37:01 I0702 08:37:01.891387 18352 hierarchical.cpp:2100] No allocations
performed
08:37:01 I0702 08:37:01.891433 18352 hierarchical.cpp:2190] No inverse offers
to send out!
08:37:01 I0702 08:37:01.891445 18352 hierarchical.cpp:1438] Performed
allocation for 1 agents in 160601ns
08:37:02 I0702 08:37:02.892103 18350 hierarchical.cpp:2100] No allocations
performed
08:37:02 I0702 08:37:02.892149 18350 hierarchical.cpp:2190] No inverse offers
to send out!
08:37:02 I0702 08:37:02.892163 18350 hierarchical.cpp:1438] Performed
allocation for 1 agents in 151446ns
08:37:06 I0702 08:37:03.892902 18352 hierarchical.cpp:2100] No allocations
performed
08:37:06 I0702 08:37:03.892947 18352 hierarchical.cpp:2190] No inverse offers
to send out!
08:37:06 I0702 08:37:03.892961 18352 hierarchical.cpp:1438] Performed
allocation for 1 agents in 143224ns
08:37:06 I0702 08:37:04.894285 18345 hierarchical.cpp:2100] No allocations
performed
08:37:06 I0702 08:37:04.894331 18345 hierarchical.cpp:2190] No inverse offers
to send out!
08:37:06 I0702 08:37:04.894345 18345 hierarchical.cpp:1438] Performed
allocation for 1 agents in 149776ns
08:37:06 I0702 08:37:05.895040 18349 hierarchical.cpp:2100] No allocations
performed
08:37:06 I0702 08:37:05.895088 18349 hierarchical.cpp:2190] No inverse offers
to send out!
08:37:06 I0702 08:37:05.895102 18349 hierarchical.cpp:1438] Performed
allocation for 1 agents in 178089ns
08:37:06 I0702 08:37:06.895756 18346 hierarchical.cpp:2100] No allocations
performed
08:37:06 I0702 08:37:06.895802 18346 hierarchical.cpp:2190] No inverse offers
to send out!
08:37:06 I0702 08:37:06.895814 18346 hierarchical.cpp:1438] Performed
allocation for 1 agents in 149783ns
08:37:07 I0702 08:37:07.896450 18347 hierarchical.cpp:2100] No allocations
performed
08:37:07 I0702 08:37:07.896497 18347 hierarchical.cpp:2190] No inverse offers
to send out!
08:37:07 I0702 08:37:07.896509 18347 hierarchical.cpp:1438] Performed
allocation for 1 agents in 154240ns
08:37:08 I0702 08:37:08.897680 18351 hierarchical.cpp:2100] No allocations
performed
08:37:08 I0702 08:37:08.897725 18351 hierarchical.cpp:2190] No inverse offers
to send out!
08:37:08 I0702 08:37:08.897739 18351 hierarchical.cpp:1438] Performed
allocation for 1 agents in 161949ns
08:37:09 I0702 08:37:09.898281 18348 hierarchical.cpp:2100] No allocations
performed
08:37:09 I0702 08:37:09.898326 18348 hierarchical.cpp:2190] No inverse offers
to send out!
08:37:09 I0702 08:37:09.898339 18348 hierarchical.cpp:1438] Performed
allocation for 1 agents in 152873ns
08:37:10 I0702 08:37:10.898883 18350 hierarchical.cpp:2100] No allocations
performed
08:37:10 I0702 08:37:10.898931 18350 hierarchical.cpp:2190] No inverse offers
to send out!
08:37:10 I0702 08:37:10.898944 18350 hierarchical.cpp:1438] Performed
allocation for 1 agents in 152239ns
08:37:11 I0702 08:37:11.899273 18352 hierarchical.cpp:2100] No allocations
performed
08:37:11 I0702 08:37:11.899319 18352 hierarchical.cpp:2190] No inverse offers
to send out!
08:37:11 I0702 08:37:11.899333 18352 hierarchical.cpp:1438] Performed
allocation for 1 agents in 148464ns
08:37:12 I0702 08:37:12.008049 18348 master.hpp:2217] Sending heartbeat to
1b41a301-e70a-4aec-830f-7850032b28c3-0000
08:37:12 I0702 08:37:12.008471 18347 scheduler.cpp:676] Enqueuing event
HEARTBEAT received from http://172.16.10.112:35906/master/api/v1/scheduler
08:37:12 ../../src/tests/default_executor_tests.cpp:1420: Failure
08:37:12 Failed to wait 15secs for subscribed
08:37:12 ../../src/tests/default_executor_tests.cpp:1410: Failure
08:37:12 Actual function call count doesn't match EXPECT_CALL(*scheduler,
subscribed(_, _))...
08:37:12 Expected: to be called once
08:37:12 Actual: never called - unsatisfied and active
08:37:12 I0702 08:37:12.009460 28719 slave.cpp:843] Agent terminating
08:37:12 I0702 08:37:12.009508 18352 master.cpp:1432] Framework
1b41a301-e70a-4aec-830f-7850032b28c3-0000 (default) disconnected
08:37:12 I0702 08:37:12.009521 18352 master.cpp:3271] Deactivating framework
1b41a301-e70a-4aec-830f-7850032b28c3-0000 (default)
08:37:12 W0702 08:37:12.009644 18352 master.hpp:2378] Unable to send event to
framework 1b41a301-e70a-4aec-830f-7850032b28c3-0000 (default): connection closed
08:37:12 I0702 08:37:12.009656 18352 master.cpp:9201] Removing offer
1b41a301-e70a-4aec-830f-7850032b28c3-O0
08:37:12 I0702 08:37:12.009672 18352 master.cpp:3248] Disconnecting framework
1b41a301-e70a-4aec-830f-7850032b28c3-0000 (default)
08:37:12 I0702 08:37:12.009682 18352 master.cpp:1447] Giving framework
1b41a301-e70a-4aec-830f-7850032b28c3-0000 (default) 0ns to failover
08:37:12 I0702 08:37:12.009723 18352 hierarchical.cpp:399] Deactivated
framework 1b41a301-e70a-4aec-830f-7850032b28c3-0000
08:37:12 I0702 08:37:12.009850 18352 hierarchical.cpp:1122] Recovered
cpus(allocated: *):2; mem(allocated: *):1024; disk(allocated: *):1024;
ports(allocated: *):[31000-32000] (total: cpus:2; mem:1024; disk:1024;
ports:[31000-32000], allocated: {}) on agent
1b41a301-e70a-4aec-830f-7850032b28c3-S0 from framework
1b41a301-e70a-4aec-830f-7850032b28c3-0000
08:37:12 I0702 08:37:12.010066 18352 master.cpp:1318] Agent
1b41a301-e70a-4aec-830f-7850032b28c3-S0 at slave(820)@172.16.10.112:35906
(ip-172-16-10-112.ec2.internal) disconnected
08:37:12 I0702 08:37:12.010089 18352 master.cpp:3308] Disconnecting agent
1b41a301-e70a-4aec-830f-7850032b28c3-S0 at slave(820)@172.16.10.112:35906
(ip-172-16-10-112.ec2.internal)
08:37:12 I0702 08:37:12.010110 18352 master.cpp:3327] Deactivating agent
1b41a301-e70a-4aec-830f-7850032b28c3-S0 at slave(820)@172.16.10.112:35906
(ip-172-16-10-112.ec2.internal)
08:37:12 I0702 08:37:12.010140 18347 hierarchical.cpp:669] Agent
1b41a301-e70a-4aec-830f-7850032b28c3-S0 deactivated
08:37:12 I0702 08:37:12.012230 18345 master.cpp:1160] Master terminating
08:37:12 I0702 08:37:12.012362 18348 hierarchical.cpp:605] Removed agent
1b41a301-e70a-4aec-830f-7850032b28c3-S0
08:37:12 I0702 08:37:12.012493 18348 hierarchical.cpp:342] Removed framework
1b41a301-e70a-4aec-830f-7850032b28c3-0000
08:37:12 [ FAILED ]
ROOT_DOCKER_DockerAndMesosContainerizers/DefaultExecutorTest.ROOT_MultiTaskgroupSharePidNamespace/0,
where GetParam() = "docker,mesos" (15135 ms){noformat}
> DefaultExecutorTest.ROOT_MultiTaskgroupSharePidNamespace is flaky.
> ------------------------------------------------------------------
>
> Key: MESOS-8305
> URL: https://issues.apache.org/jira/browse/MESOS-8305
> Project: Mesos
> Issue Type: Bug
> Affects Versions: 1.4.1
> Environment: Ubuntu 16.04
> Fedora 23
> Reporter: Alexander Rukletsov
> Assignee: Qian Zhang
> Priority: Major
> Labels: flaky-test
> Fix For: 1.6.0
>
> Attachments: ROOT_MultiTaskgroupSharePidNamespace-badrun.txt
>
>
> On Ubuntu 16.04:
> {noformat}
> ../../src/tests/default_executor_tests.cpp:1877
> Expected: strings::trim(pidNamespace1.get())
> Which is: "4026532250"
> To be equal to: strings::trim(pidNamespace2.get())
> Which is: ""
> {noformat}
> Full log attached.
> On Fedora 23:
> {noformat}
> ../../src/tests/default_executor_tests.cpp:1878
> Expected: strings::trim(pidNamespace1.get())
> Which is: "4026532233"
> To be equal to: strings::trim(pidNamespace2.get())
> Which is: ""
> {noformat}
> The test became flaky shortly after MESOS-7306 has been committed and likely
> related to it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
