[
https://issues.apache.org/jira/browse/MESOS-6345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15559169#comment-15559169
]
Avinash Sridharan commented on MESOS-6345:
------------------------------------------
```
[04:56:48] : [Step 10/10] [ RUN ]
ExamplesTest.PersistentVolumeFramework
[04:56:48]W: [Step 10/10] I1008 04:56:48.212661 25257 master.cpp:1097]
Master terminating
[04:56:48]W: [Step 10/10] I1008 04:56:48.212674 25254
status_update_manager.cpp:395] Received status update acknowledgement (UUID:
542b14f7-bfc9-4be3-81b4-c23a1da9ecb5) for task 2 of framework
84cfc7a4-ad66-4f0d-965c-33ff6093ef32-0000
[04:56:48]W: [Step 10/10] I1008 04:56:48.212709 25254
status_update_manager.cpp:531] Cleaning up status update stream for task 2 of
framework 84cfc7a4-ad66-4f0d-965c-33ff6093ef32-0000
[04:56:48]W: [Step 10/10] I1008 04:56:48.212712 25257 master.cpp:7725]
Removing executor 'default' with resources {} of framework
84cfc7a4-ad66-4f0d-965c-33ff6093ef32-0000 on agent
84cfc7a4-ad66-4f0d-965c-33ff6093ef32-S2 at slave(1)@172.30.2.21:52703
(ip-172-30-2-21.mesosphere.io)
[04:56:48]W: [Step 10/10] I1008 04:56:48.212767 25254 slave.cpp:2953]
Status update manager successfully handled status update acknowledgement (UUID:
542b14f7-bfc9-4be3-81b4-c23a1da9ecb5) for task 2 of framework
84cfc7a4-ad66-4f0d-965c-33ff6093ef32-0000
[04:56:48]W: [Step 10/10] I1008 04:56:48.212782 25254 slave.cpp:6543]
Completing task 2
[04:56:48]W: [Step 10/10] I1008 04:56:48.212792 25258 hierarchical.cpp:517]
Removed agent 84cfc7a4-ad66-4f0d-965c-33ff6093ef32-S2
[04:56:48]W: [Step 10/10] I1008 04:56:48.212829 25257 master.cpp:7696]
Removing task 3 with resources cpus(*):1; mem(*):128 of framework
84cfc7a4-ad66-4f0d-965c-33ff6093ef32-0000 on agent
84cfc7a4-ad66-4f0d-965c-33ff6093ef32-S1 at slave(3)@172.30.2.21:52703
(ip-172-30-2-21.mesosphere.io)
[04:56:48]W: [Step 10/10] I1008 04:56:48.212888 25257 master.cpp:7725]
Removing executor 'default' with resources {} of framework
84cfc7a4-ad66-4f0d-965c-33ff6093ef32-0000 on agent
84cfc7a4-ad66-4f0d-965c-33ff6093ef32-S1 at slave(3)@172.30.2.21:52703
(ip-172-30-2-21.mesosphere.io)
[04:56:48]W: [Step 10/10] I1008 04:56:48.212915 25258 hierarchical.cpp:517]
Removed agent 84cfc7a4-ad66-4f0d-965c-33ff6093ef32-S1
[04:56:48]W: [Step 10/10] I1008 04:56:48.213017 25257 master.cpp:7725]
Removing executor 'default' with resources {} of framework
84cfc7a4-ad66-4f0d-965c-33ff6093ef32-0000 on agent
84cfc7a4-ad66-4f0d-965c-33ff6093ef32-S0 at slave(2)@172.30.2.21:52703
(ip-172-30-2-21.mesosphere.io)
[04:56:48]W: [Step 10/10] I1008 04:56:48.213102 25254 hierarchical.cpp:517]
Removed agent 84cfc7a4-ad66-4f0d-965c-33ff6093ef32-S0
[04:56:48]W: [Step 10/10] I1008 04:56:48.213281 25251 hierarchical.cpp:337]
Removed framework 84cfc7a4-ad66-4f0d-965c-33ff6093ef32-0000
[04:56:48]W: [Step 10/10] I1008 04:56:48.213404 25257 slave.cpp:4174] Got
exited event for [email protected]:52703
[04:56:48]W: [Step 10/10] I1008 04:56:48.213418 25253 slave.cpp:4174] Got
exited event for [email protected]:52703
[04:56:48]W: [Step 10/10] W1008 04:56:48.213426 25257 slave.cpp:4179]
Master disconnected! Waiting for a new master to be elected
[04:56:48]W: [Step 10/10] W1008 04:56:48.213433 25253 slave.cpp:4179]
Master disconnected! Waiting for a new master to be elected
[04:56:48]W: [Step 10/10] I1008 04:56:48.213407 25254 slave.cpp:4174] Got
exited event for [email protected]:52703
[04:56:48]W: [Step 10/10] W1008 04:56:48.213448 25254 slave.cpp:4179]
Master disconnected! Waiting for a new master to be elected
[04:56:48]W: [Step 10/10] I1008 04:56:48.214047 25254 slave.cpp:787] Agent
terminating
[04:56:48]W: [Step 10/10] I1008 04:56:48.214068 25254 slave.cpp:2506] Asked
to shut down framework 84cfc7a4-ad66-4f0d-965c-33ff6093ef32-0000 by @0.0.0.0:0
[04:56:48]W: [Step 10/10] I1008 04:56:48.214076 25254 slave.cpp:2531]
Shutting down framework 84cfc7a4-ad66-4f0d-965c-33ff6093ef32-0000
[04:56:48]W: [Step 10/10] I1008 04:56:48.214083 25254 slave.cpp:4855]
Shutting down executor 'default' of framework
84cfc7a4-ad66-4f0d-965c-33ff6093ef32-0000 (via HTTP)
[04:56:48]W: [Step 10/10] E1008 04:56:48.215160 25384 executor.cpp:681]
End-Of-File received from agent. The agent closed the event stream
[04:56:48]W: [Step 10/10] I1008 04:56:48.215250 25254 slave.cpp:787] Agent
terminating
[04:56:48]W: [Step 10/10] I1008 04:56:48.215266 25254 slave.cpp:2506] Asked
to shut down framework 84cfc7a4-ad66-4f0d-965c-33ff6093ef32-0000 by @0.0.0.0:0
[04:56:48]W: [Step 10/10] I1008 04:56:48.215279 25254 slave.cpp:2531]
Shutting down framework 84cfc7a4-ad66-4f0d-965c-33ff6093ef32-0000
[04:56:48]W: [Step 10/10] I1008 04:56:48.215291 25254 slave.cpp:4855]
Shutting down executor 'default' of framework
84cfc7a4-ad66-4f0d-965c-33ff6093ef32-0000 (via HTTP)
[04:56:48]W: [Step 10/10] E1008 04:56:48.216333 25366 executor.cpp:681]
End-Of-File received from agent. The agent closed the event stream
[04:56:48]W: [Step 10/10] I1008 04:56:48.216490 25226 slave.cpp:787] Agent
terminating
[04:56:48]W: [Step 10/10] I1008 04:56:48.216507 25226 slave.cpp:2506] Asked
to shut down framework 84cfc7a4-ad66-4f0d-965c-33ff6093ef32-0000 by @0.0.0.0:0
[04:56:48]W: [Step 10/10] I1008 04:56:48.216518 25226 slave.cpp:2531]
Shutting down framework 84cfc7a4-ad66-4f0d-965c-33ff6093ef32-0000
[04:56:48]W: [Step 10/10] I1008 04:56:48.216524 25226 slave.cpp:4855]
Shutting down executor 'default' of framework
84cfc7a4-ad66-4f0d-965c-33ff6093ef32-0000 (via HTTP)
[04:56:48]W: [Step 10/10] E1008 04:56:48.217491 25387 executor.cpp:681]
End-Of-File received from agent. The agent closed the event stream
[04:56:48]W: [Step 10/10] Using temporary directory
'/mnt/teamcity/temp/buildTmp/ExamplesTest_PersistentVolumeFramework_zqfSTm'
[04:56:48]W: [Step 10/10]
/mnt/teamcity/work/4240ba9ddd0997c3/src/tests/persistent_volume_framework_test.sh:
line 20: /mnt/teamcity/work/4240ba9ddd0997c3/build/src/colors.sh: No such file
or directory
[04:56:48]W: [Step 10/10]
/mnt/teamcity/work/4240ba9ddd0997c3/src/tests/persistent_volume_framework_test.sh:
line 21: /mnt/teamcity/work/4240ba9ddd0997c3/build/src/atexit.sh: No such file
or directory
[04:56:48]W: [Step 10/10] I1008 04:56:48.558832 25396 logging.cpp:199]
Logging to STDERR
[04:56:48]W: [Step 10/10] I1008 04:56:48.561102 25396 process.cpp:1069]
libprocess is initialized on 172.30.2.21:45167 with 8 worker threads
[04:56:48]W: [Step 10/10] I1008 04:56:48.575021 25396 leveldb.cpp:174]
Opened db in 12.462333ms
[04:56:48]W: [Step 10/10] I1008 04:56:48.582849 25396 leveldb.cpp:181]
Compacted db in 7.809821ms
[04:56:48]W: [Step 10/10] I1008 04:56:48.582872 25396 leveldb.cpp:196]
Created db iterator in 8910ns
[04:56:48]W: [Step 10/10] I1008 04:56:48.582878 25396 leveldb.cpp:202]
Seeked to beginning of db in 807ns
[04:56:48]W: [Step 10/10] I1008 04:56:48.582882 25396 leveldb.cpp:271]
Iterated through 0 keys in the db in 418ns
[04:56:48]W: [Step 10/10] I1008 04:56:48.582906 25396 replica.cpp:776]
Replica recovered with log positions 0 -> 0 with 1 holes and 0 unlearned
[04:56:48]W: [Step 10/10] I1008 04:56:48.583354 25423 recover.cpp:451]
Starting replica recovery
[04:56:48]W: [Step 10/10] I1008 04:56:48.583447 25425 recover.cpp:477]
Replica is in EMPTY status
[04:56:48]W: [Step 10/10] I1008 04:56:48.583479 25396 local.cpp:255]
Creating default 'local' authorizer
[04:56:48]W: [Step 10/10] I1008 04:56:48.583894 25421 replica.cpp:673]
Replica in EMPTY status received a broadcasted recover request from
__req_res__(1)@172.30.2.21:45167
[04:56:48]W: [Step 10/10] I1008 04:56:48.584003 25421 recover.cpp:197]
Received a recover response from a replica in EMPTY status
[04:56:48]W: [Step 10/10] I1008 04:56:48.584157 25426 recover.cpp:568]
Updating replica status to STARTING
[04:56:48]W: [Step 10/10] I1008 04:56:48.584314 25421 master.cpp:380]
Master c5e5ed69-9ab4-460d-99c8-d613dca680a9 (ip-172-30-2-21.mesosphere.io)
started on 172.30.2.21:45167
[04:56:48]W: [Step 10/10] I1008 04:56:48.584326 25421 master.cpp:382] Flags
at startup: --acls="permissive: true
[04:56:48]W: [Step 10/10] register_frameworks {
[04:56:48]W: [Step 10/10] principals {
[04:56:48]W: [Step 10/10] type: ANY
[04:56:48]W: [Step 10/10] }
[04:56:48]W: [Step 10/10] roles {
[04:56:48]W: [Step 10/10] type: SOME
[04:56:48]W: [Step 10/10] values: "test"
[04:56:48]W: [Step 10/10] }
[04:56:48]W: [Step 10/10] }
[04:56:48]W: [Step 10/10] " --agent_ping_timeout="15secs"
--agent_reregister_timeout="10mins" --allocation_interval="1secs"
--allocator="HierarchicalDRF" --authenticate_agents="false"
--authenticate_frameworks="false" --authenticate_http_frameworks="false"
--authenticate_http_readonly="false" --authenticate_http_readwrite="false"
--authenticators="crammd5" --authorizers="local"
--credentials="/mnt/teamcity/temp/buildTmp/ExamplesTest_PersistentVolumeFramework_zqfSTm/credentials"
--framework_sorter="drf" --help="true" --hostname_lookup="true"
--http_authenticators="basic" --initialize_driver_logging="true"
--log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
--max_agent_ping_timeouts="5" --max_completed_frameworks="50"
--max_completed_tasks_per_framework="1000" --quiet="false"
--recovery_agent_removal_limit="100%" --registry="replicated_log"
--registry_fetch_timeout="1mins" --registry_gc_interval="15mins"
--registry_max_agent_age="2weeks" --registry_max_agent_count="102400"
--registry_store_timeout="20secs" --registry_strict="false" --roles="test"
--root_submissions="true" --user_sorter="drf" --version="false"
--webui_dir="/mnt/teamcity/work/4240ba9ddd0997c3/src/webui"
--work_dir="/mnt/teamcity/temp/buildTmp/mesos-8KiPML"
--zk_session_timeout="10secs"
[04:56:48]W: [Step 10/10] I1008 04:56:48.584619 25421 master.cpp:434]
Master allowing unauthenticated frameworks to register
[04:56:48]W: [Step 10/10] I1008 04:56:48.584630 25421 master.cpp:448]
Master allowing unauthenticated agents to register
[04:56:48]W: [Step 10/10] I1008 04:56:48.584636 25421 master.cpp:462]
Master allowing HTTP frameworks to register without authentication
[04:56:48]W: [Step 10/10] I1008 04:56:48.584645 25421 credentials.hpp:37]
Loading credentials for authentication from
'/mnt/teamcity/temp/buildTmp/ExamplesTest_PersistentVolumeFramework_zqfSTm/credentials'
[04:56:48]W: [Step 10/10] I1008 04:56:48.584748 25421 master.cpp:504] Using
default 'crammd5' authenticator
[04:56:48]W: [Step 10/10] I1008 04:56:48.584777 25421
authenticator.cpp:519] Initializing server SASL
[04:56:48]W: [Step 10/10] I1008 04:56:48.585579 25396
containerizer.cpp:202] Using isolation:
filesystem/posix,posix/cpu,posix/mem,network/cni
[04:56:48]W: [Step 10/10] I1008 04:56:48.587158 25426 slave.cpp:208] Mesos
agent started on (1)@172.30.2.21:45167
[04:56:48]W: [Step 10/10] I1008 04:56:48.587236 25421 auxprop.cpp:73]
Initialized in-memory auxiliary property plugin
[04:56:48]W: [Step 10/10] I1008 04:56:48.587273 25421 master.cpp:584]
Authorization enabled
[04:56:48]W: [Step 10/10] W1008 04:56:48.587286 25421 master.cpp:647] The
'--roles' flag is deprecated. This flag will be removed in the future. See the
Mesos 0.27 upgrade notes for more information
[04:56:48]W: [Step 10/10] I1008 04:56:48.587389 25422 hierarchical.cpp:149]
Initialized hierarchical allocator process
[04:56:48]W: [Step 10/10] I1008 04:56:48.587393 25427
whitelist_watcher.cpp:77] No whitelist given
[04:56:48]W: [Step 10/10] I1008 04:56:48.587170 25426 slave.cpp:209] Flags
at startup: --acls="permissive: true
[04:56:48]W: [Step 10/10] register_frameworks {
[04:56:48]W: [Step 10/10] principals {
[04:56:48]W: [Step 10/10] type: ANY
[04:56:48]W: [Step 10/10] }
[04:56:48]W: [Step 10/10] roles {
[04:56:48]W: [Step 10/10] type: SOME
[04:56:48]W: [Step 10/10] values: "test"
[04:56:48]W: [Step 10/10] }
[04:56:48]W: [Step 10/10] }
[04:56:48]W: [Step 10/10] " --appc_simple_discovery_uri_prefix="http://";
--appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="false"
--authenticate_http_readwrite="false" --authenticatee="crammd5"
--authentication_backoff_factor="1secs" --authorizer="local"
--cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false"
--cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false"
--cgroups_root="mesos" --container_disk_watch_interval="15secs"
--containerizers="mesos" --default_role="test" --disk_watch_interval="1mins"
--docker="docker" --docker_kill_orphans="true"
--docker_registry="https://registry-1.docker.io"; --docker_remove_delay="6hrs"
--docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns"
--docker_store_dir="/tmp/mesos/store/docker"
--docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume"
--enforce_container_disk_quota="false" --executor_registration_timeout="1mins"
--executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/mesos/fetch"
--fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks"
--gc_disk_headroom="0.1" --hadoop_home="" --help="false"
--hostname_lookup="true" --http_authenticators="basic"
--http_command_executor="false" --image_provisioner_backend="copy"
--initialize_driver_logging="true"
--isolation="filesystem/posix,posix/cpu,posix/mem" --launcher="posix"
--launcher_dir="/mnt/teamcity/work/4240ba9ddd0997c3/build/src" --logbufsecs="0"
--logging_level="INFO" --oversubscribed_resources_interval="15secs"
--perf_duration="10secs" --perf_interval="1mins"
--qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect"
--recovery_timeout="15mins" --registration_backoff_factor="1secs"
--revocable_cpu_low_priority="true"
--runtime_dir="/mnt/teamcity/temp/buildTmp/mesos-gkgAKP"
--sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true"
--systemd_enable_support="true"
--systemd_runtime_directory="/run/systemd/system" --version="false"
--work_dir="/mnt/teamcity/temp/buildTmp/mesos-8KiPML/0"
[04:56:48]W: [Step 10/10] I1008 04:56:48.587808 25426 slave.cpp:533] Agent
resources: cpus(test):8; mem(test):14015; disk(test):75375;
ports(test):[31000-32000]
[04:56:48]W: [Step 10/10] I1008 04:56:48.587857 25426 slave.cpp:541] Agent
attributes: [ ]
[04:56:48]W: [Step 10/10] I1008 04:56:48.587870 25426 slave.cpp:546] Agent
hostname: ip-172-30-2-21.mesosphere.io
[04:56:48]W: [Step 10/10] I1008 04:56:48.588235 25396
containerizer.cpp:202] Using isolation:
filesystem/posix,posix/cpu,posix/mem,network/cni
[04:56:48]W: [Step 10/10] I1008 04:56:48.588538 25428 leveldb.cpp:304]
Persisting metadata (8 bytes) to leveldb took 4.284018ms
[04:56:48]W: [Step 10/10] I1008 04:56:48.588562 25428 replica.cpp:320]
Persisted replica status to STARTING
[04:56:48]W: [Step 10/10] I1008 04:56:48.588670 25425 recover.cpp:477]
Replica is in STARTING status
[04:56:48]W: [Step 10/10] I1008 04:56:48.588866 25424 replica.cpp:673]
Replica in STARTING status received a broadcasted recover request from
__req_res__(2)@172.30.2.21:45167
[04:56:48]W: [Step 10/10] I1008 04:56:48.588955 25424 recover.cpp:197]
Received a recover response from a replica in STARTING status
[04:56:48]W: [Step 10/10] I1008 04:56:48.589053 25428 recover.cpp:568]
Updating replica status to VOTING
[04:56:48]W: [Step 10/10] I1008 04:56:48.589443 25423 slave.cpp:208] Mesos
agent started on (2)@172.30.2.21:45167
[04:56:48]W: [Step 10/10] I1008 04:56:48.589452 25423 slave.cpp:209] Flags
at startup: --acls="permissive: true
[04:56:48]W: [Step 10/10] register_frameworks {
[04:56:48]W: [Step 10/10] principals {
[04:56:48]W: [Step 10/10] type: ANY
[04:56:48]W: [Step 10/10] }
[04:56:48]W: [Step 10/10] roles {
[04:56:48]W: [Step 10/10] type: SOME
[04:56:48]W: [Step 10/10] values: "test"
[04:56:48]W: [Step 10/10] }
[04:56:48]W: [Step 10/10] }
[04:56:48]W: [Step 10/10] " --appc_simple_discovery_uri_prefix="http://";
--appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="false"
--authenticate_http_readwrite="false" --authenticatee="crammd5"
--authentication_backoff_factor="1secs" --authorizer="local"
--cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false"
--cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false"
--cgroups_root="mesos" --container_disk_watch_interval="15secs"
--containerizers="mesos" --default_role="test" --disk_watch_interval="1mins"
--docker="docker" --docker_kill_orphans="true"
--docker_registry="https://registry-1.docker.io"; --docker_remove_delay="6hrs"
--docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns"
--docker_store_dir="/tmp/mesos/store/docker"
--docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume"
--enforce_container_disk_quota="false" --executor_registration_timeout="1mins"
--executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/mesos/fetch"
--fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks"
--gc_disk_headroom="0.1" --hadoop_home="" --help="false"
--hostname_lookup="true" --http_authenticators="basic"
--http_command_executor="false" --image_provisioner_backend="copy"
--initialize_driver_logging="true"
--isolation="filesystem/posix,posix/cpu,posix/mem" --launcher="posix"
--launcher_dir="/mnt/teamcity/work/4240ba9ddd0997c3/build/src" --logbufsecs="0"
--logging_level="INFO" --oversubscribed_resources_interval="15secs"
--perf_duration="10secs" --perf_interval="1mins"
--qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect"
--recovery_timeout="15mins" --registration_backoff_factor="1secs"
--revocable_cpu_low_priority="true"
--runtime_dir="/mnt/teamcity/temp/buildTmp/mesos-gkgAKP"
--sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true"
--systemd_enable_support="true"
--systemd_runtime_directory="/run/systemd/system" --version="false"
--work_dir="/mnt/teamcity/temp/buildTmp/mesos-8KiPML/1"
[04:56:48]W: [Step 10/10] I1008 04:56:48.589845 25423 slave.cpp:533] Agent
resources: cpus(test):8; mem(test):14015; disk(test):75375;
ports(test):[31000-32000]
[04:56:48]W: [Step 10/10] I1008 04:56:48.589870 25423 slave.cpp:541] Agent
attributes: [ ]
[04:56:48]W: [Step 10/10] I1008 04:56:48.589874 25423 slave.cpp:546] Agent
hostname: ip-172-30-2-21.mesosphere.io
[04:56:48]W: [Step 10/10] I1008 04:56:48.590322 25396
containerizer.cpp:202] Using isolation:
filesystem/posix,posix/cpu,posix/mem,network/cni
[04:56:48]W: [Step 10/10] I1008 04:56:48.591367 25424 slave.cpp:208] Mesos
agent started on (3)@172.30.2.21:45167
[04:56:48]W: [Step 10/10] I1008 04:56:48.591378 25424 slave.cpp:209] Flags
at startup: --acls="permissive: true
[04:56:48]W: [Step 10/10] register_frameworks {
[04:56:48]W: [Step 10/10] principals {
[04:56:48]W: [Step 10/10] type: ANY
[04:56:48]W: [Step 10/10] }
[04:56:48]W: [Step 10/10] roles {
[04:56:48]W: [Step 10/10] type: SOME
[04:56:48]W: [Step 10/10] values: "test"
[04:56:48]W: [Step 10/10] }
[04:56:48]W: [Step 10/10] }
[04:56:48]W: [Step 10/10] " --appc_simple_discovery_uri_prefix="http://";
--appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="false"
--authenticate_http_readwrite="false" --authenticatee="crammd5"
--authentication_backoff_factor="1secs" --authorizer="local"
--cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false"
--cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false"
--cgroups_root="mesos" --container_disk_watch_interval="15secs"
--containerizers="mesos" --default_role="test" --disk_watch_interval="1mins"
--docker="docker" --docker_kill_orphans="true"
--docker_registry="https://registry-1.docker.io"; --docker_remove_delay="6hrs"
--docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns"
--docker_store_dir="/tmp/mesos/store/docker"
--docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume"
--enforce_container_disk_quota="false" --executor_registration_timeout="1mins"
--executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/mesos/fetch"
--fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks"
--gc_disk_headroom="0.1" --hadoop_home="" --help="true"
--hostname_lookup="true" --http_authenticators="basic"
--http_command_executor="false" --image_provisioner_backend="copy"
--initialize_driver_logging="true"
--isolation="filesystem/posix,posix/cpu,posix/mem" --launcher="posix"
--launcher_dir="/mnt/teamcity/work/4240ba9ddd0997c3/build/src" --logbufsecs="0"
--logging_level="INFO" --oversubscribed_resources_interval="15secs"
--perf_duration="10secs" --perf_interval="1mins"
--qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect"
--recovery_timeout="15mins" --registration_backoff_factor="1secs"
--revocable_cpu_low_priority="true"
--runtime_dir="/mnt/teamcity/temp/buildTmp/mesos-gkgAKP"
--sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true"
--systemd_enable_support="true"
--systemd_runtime_directory="/run/systemd/system" --version="false"
--work_dir="/mnt/teamcity/temp/buildTmp/mesos-8KiPML/2"
[04:56:48]W: [Step 10/10] I1008 04:56:48.591719 25424 slave.cpp:533] Agent
resources: cpus(test):8; mem(test):14015; disk(test):75375;
ports(test):[31000-32000]
[04:56:48]W: [Step 10/10] I1008 04:56:48.591742 25424 slave.cpp:541] Agent
attributes: [ ]
[04:56:48]W: [Step 10/10] I1008 04:56:48.591744 25424 slave.cpp:546] Agent
hostname: ip-172-30-2-21.mesosphere.io
[04:56:48]W: [Step 10/10] *** Error in
`/mnt/teamcity/work/4240ba9ddd0997c3/build/src/.libs/lt-persistent-volume-framework':
double free or corruption (fasttop): 0x00007f1ae0006a20 ***
[04:56:48]W: [Step 10/10] *** Aborted at 1475902608 (unix time) try "date
-d @1475902608" if you are using GNU date ***
[04:56:48]W: [Step 10/10] I1008 04:56:48.592744 25425 state.cpp:57]
Recovering state from '/mnt/teamcity/temp/buildTmp/mesos-8KiPML/2/meta'
[04:56:48]W: [Step 10/10] I1008 04:56:48.592808 25423 state.cpp:57]
Recovering state from '/mnt/teamcity/temp/buildTmp/mesos-8KiPML/1/meta'
[04:56:48]W: [Step 10/10] I1008 04:56:48.592952 25425
status_update_manager.cpp:203] Recovering status update manager
[04:56:48]W: [Step 10/10] I1008 04:56:48.592957 25423
status_update_manager.cpp:203] Recovering status update manager
[04:56:48]W: [Step 10/10] I1008 04:56:48.593010 25424
containerizer.cpp:557] Recovering containerizer
[04:56:48]W: [Step 10/10] I1008 04:56:48.593143 25396 sched.cpp:226]
Version: 1.1.0
[04:56:48]W: [Step 10/10] I1008 04:56:48.593158 25425 master.cpp:2013]
Elected as the leading master!
[04:56:48]W: [Step 10/10] I1008 04:56:48.593173 25425 master.cpp:1560]
Recovering from registrar
[04:56:48]W: [Step 10/10] I1008 04:56:48.593211 25424 registrar.cpp:329]
Recovering registrar
[04:56:48]W: [Step 10/10] I1008 04:56:48.593250 25425 sched.cpp:330] New
master detected at [email protected]:45167
[04:56:48]W: [Step 10/10] I1008 04:56:48.593282 25425 sched.cpp:341] No
credentials provided. Attempting to register without authentication
[04:56:48]W: [Step 10/10] I1008 04:56:48.593293 25425 sched.cpp:820]
Sending SUBSCRIBE call to [email protected]:45167
[04:56:48]W: [Step 10/10] PC: @ 0x7f1b0bbaccc9 (unknown)
[04:56:48]W: [Step 10/10] I1008 04:56:48.593339 25425 sched.cpp:853] Will
retry registration in 32.354951ms if necessary
[04:56:48]W: [Step 10/10] I1008 04:56:48.593364 25421 master.cpp:1387]
Dropping 'mesos.scheduler.Call' message since not recovered yet
[04:56:48]W: [Step 10/10] I1008 04:56:48.593413 25428 provisioner.cpp:253]
Provisioner recovery complete
[04:56:48]W: [Step 10/10] *** SIGABRT (@0x6334) received by PID 25396 (TID
0x7f1b02ed6700) from PID 25396; stack trace: ***
[04:56:48]W: [Step 10/10] I1008 04:56:48.593520 25421
containerizer.cpp:557] Recovering containerizer
[04:56:48]W: [Step 10/10] I1008 04:56:48.593529 25425 slave.cpp:5276]
Finished recovery
[04:56:48]W: [Step 10/10] I1008 04:56:48.593627 25422 leveldb.cpp:304]
Persisting metadata (8 bytes) to leveldb took 4.546422ms
[04:56:48]W: [Step 10/10] I1008 04:56:48.593695 25428 provisioner.cpp:253]
Provisioner recovery complete
[04:56:48]W: [Step 10/10] I1008 04:56:48.593701 25422 replica.cpp:320]
Persisted replica status to VOTING
[04:56:48]W: [Step 10/10] I1008 04:56:48.593760 25424 slave.cpp:5276]
Finished recovery
[04:56:48]W: [Step 10/10] I1008 04:56:48.593864 25427 recover.cpp:582]
Successfully joined the Paxos group
[04:56:48]W: [Step 10/10] I1008 04:56:48.593896 25425 slave.cpp:5448]
Querying resource estimator for oversubscribable resources
[04:56:48]W: [Step 10/10] I1008 04:56:48.593922 25427 recover.cpp:466]
Recover process terminated
[04:56:48]W: [Step 10/10] I1008 04:56:48.593976 25427 slave.cpp:5462]
Received oversubscribable resources {} from the resource estimator
[04:56:48]W: [Step 10/10] I1008 04:56:48.594002 25424 slave.cpp:5448]
Querying resource estimator for oversubscribable resources
[04:56:48]W: [Step 10/10] I1008 04:56:48.594017 25422 log.cpp:553]
Attempting to start the writer
[04:56:48]W: [Step 10/10] I1008 04:56:48.594030 25428
status_update_manager.cpp:177] Pausing sending status updates
[04:56:48]W: [Step 10/10] I1008 04:56:48.594032 25427 slave.cpp:915] New
master detected at [email protected]:45167
[04:56:48]W: [Step 10/10] I1008 04:56:48.594055 25423 slave.cpp:915] New
master detected at [email protected]:45167
[04:56:48]W: [Step 10/10] I1008 04:56:48.594048 25428
status_update_manager.cpp:177] Pausing sending status updates
[04:56:48]W: [Step 10/10] I1008 04:56:48.594061 25427 slave.cpp:936] No
credentials provided. Attempting to register without authentication
[04:56:48]W: [Step 10/10] I1008 04:56:48.594106 25427 slave.cpp:947]
Detecting new master
[04:56:48]W: [Step 10/10] I1008 04:56:48.594071 25423 slave.cpp:936] No
credentials provided. Attempting to register without authentication
[04:56:48]W: [Step 10/10] @ 0x7f1b0bf4b340 (unknown)
[04:56:48]W: [Step 10/10] I1008 04:56:48.594125 25423 slave.cpp:947]
Detecting new master
[04:56:48]W: [Step 10/10] I1008 04:56:48.594194 25423 slave.cpp:5462]
Received oversubscribable resources {} from the resource estimator
[04:56:48]W: [Step 10/10] I1008 04:56:48.594378 25422 replica.cpp:493]
Replica received implicit promise request from __req_res__(3)@172.30.2.21:45167
with proposal 1
[04:56:48]W: [Step 10/10] @ 0x7f1b0bbaccc9 (unknown)
[04:56:48]W: [Step 10/10] @ 0x7f1b0bbb00d8 (unknown)
[04:56:48]W: [Step 10/10] @ 0x7f1b0bbe9394 (unknown)
[04:56:48]W: [Step 10/10] I1008 04:56:48.595368 25422 leveldb.cpp:304]
Persisting metadata (8 bytes) to leveldb took 972334ns
[04:56:48]W: [Step 10/10] I1008 04:56:48.595381 25422 replica.cpp:342]
Persisted promised to 1
[04:56:48]W: [Step 10/10] @ 0x7f1b0bbf566e (unknown)
[04:56:48]W: [Step 10/10] @ 0x7f1b0d930925
_ZNSt14_Function_base13_Base_managerIZN7process5deferIN5mesos8internal5slave5SlaveEiiSt12_PlaceholderILi1EES7_ILi2EEEENS1_9_DeferredIDTcl4bindadsrSt8functionIFvT0_T1_EEclcvSF__Efp1_fp2_EEEERKNS1_3PIDIT_EEMSJ_FvSC_SD_ET2_T3_EUliiE_E10_M_managerERSt9_Any_dataRKST_St18_Manager_operation
[04:56:48]W: [Step 10/10] I1008 04:56:48.597909 25421 coordinator.cpp:238]
Coordinator attempting to fill missing positions
[04:56:48]W: [Step 10/10] I1008 04:56:48.598273 25423 replica.cpp:388]
Replica received explicit promise request from __req_res__(4)@172.30.2.21:45167
for position 0 with proposal 2
[04:56:48]W: [Step 10/10] @ 0x7f1b0d935b0b
std::_Function_base::_Base_manager<>::_M_manager()
[04:56:48]W: [Step 10/10] @ 0x7f1b0d8f2516
os::internal::configureSignal()
[04:56:48]W: [Step 10/10] I1008 04:56:48.599318 25423 leveldb.cpp:341]
Persisting action (8 bytes) to leveldb took 1.024957ms
[04:56:48]W: [Step 10/10] I1008 04:56:48.599333 25423 replica.cpp:708]
Persisted action NOP at position 0
[04:56:48]W: [Step 10/10] I1008 04:56:48.599630 25428 replica.cpp:537]
Replica received write request for position 0 from
__req_res__(5)@172.30.2.21:45167
[04:56:48]W: [Step 10/10] I1008 04:56:48.599660 25428 leveldb.cpp:436]
Reading position from leveldb took 16893ns
[04:56:48]W: [Step 10/10] @ 0x7f1b0d904022
mesos::internal::slave::Slave::initialize()
[04:56:48]W: [Step 10/10] @ 0x7f1b0e0c6ed1
process::ProcessManager::resume()
[04:56:48]W: [Step 10/10] @ 0x7f1b0e0c7187
_ZNSt6thread5_ImplISt12_Bind_simpleIFZN7process14ProcessManager12init_threadsEvEUt_vEEE6_M_runEv
[04:56:48]W: [Step 10/10] @ 0x7f1b0c726a60 (unknown)
[04:56:48]W: [Step 10/10] @ 0x7f1b0bf43182 start_thread
[04:56:48]W: [Step 10/10] @ 0x7f1b0bc7047d (unknown)
[04:56:49] : [Step 10/10] ../../src/tests/script.cpp:80: Failure
[04:56:49] : [Step 10/10] Failed
[04:56:49] : [Step 10/10] persistent_volume_framework_test.sh terminated
with signal Aborted
```
> ExamplesTest.PersistentVolumeFramework failing due to double free corruption
> on Ubuntu 14.04
> --------------------------------------------------------------------------------------------
>
> Key: MESOS-6345
> URL: https://issues.apache.org/jira/browse/MESOS-6345
> Project: Mesos
> Issue Type: Bug
> Components: framework
> Reporter: Avinash Sridharan
> Labels: mesosphere
>
> PersistentVolumeFramework tests if failing on Ubuntu 14
> {code}
> [Step 10/10] *** Error in
> `/mnt/teamcity/work/4240ba9ddd0997c3/build/src/.libs/lt-persistent-volume-framework':
> double free or corruption (fasttop): 0x00007f1ae0006a20 ***
> [04:56:48]W: [Step 10/10] *** Aborted at 1475902608 (unix time) try "date
> -d @1475902608" if you are using GNU date ***
> [04:56:48]W: [Step 10/10] I1008 04:56:48.592744 25425 state.cpp:57]
> Recovering state from '/mnt/teamcity/temp/buildTmp/mesos-8KiPML/2/meta'
> [04:56:48]W: [Step 10/10] I1008 04:56:48.592808 25423 state.cpp:57]
> Recovering state from '/mnt/teamcity/temp/buildTmp/mesos-8KiPML/1/meta'
> [04:56:48]W: [Step 10/10] I1008 04:56:48.592952 25425
> status_update_manager.cpp:203] Recovering status update manager
> [04:56:48]W: [Step 10/10] I1008 04:56:48.592957 25423
> status_update_manager.cpp:203] Recovering status update manager
> [04:56:48]W: [Step 10/10] I1008 04:56:48.593010 25424
> containerizer.cpp:557] Recovering containerizer
> [04:56:48]W: [Step 10/10] I1008 04:56:48.593143 25396 sched.cpp:226]
> Version: 1.1.0
> [04:56:48]W: [Step 10/10] I1008 04:56:48.593158 25425 master.cpp:2013]
> Elected as the leading master!
> [04:56:48]W: [Step 10/10] I1008 04:56:48.593173 25425 master.cpp:1560]
> Recovering from registrar
> [04:56:48]W: [Step 10/10] I1008 04:56:48.593211 25424 registrar.cpp:329]
> Recovering registrar
> [04:56:48]W: [Step 10/10] I1008 04:56:48.593250 25425 sched.cpp:330] New
> master detected at [email protected]:45167
> [04:56:48]W: [Step 10/10] I1008 04:56:48.593282 25425 sched.cpp:341] No
> credentials provided. Attempting to register without authentication
> [04:56:48]W: [Step 10/10] I1008 04:56:48.593293 25425 sched.cpp:820]
> Sending SUBSCRIBE call to [email protected]:45167
> [04:56:48]W: [Step 10/10] PC: @ 0x7f1b0bbaccc9 (unknown)
> [04:56:48]W: [Step 10/10] I1008 04:56:48.593339 25425 sched.cpp:853] Will
> retry registration in 32.354951ms if necessary
> [04:56:48]W: [Step 10/10] I1008 04:56:48.593364 25421 master.cpp:1387]
> Dropping 'mesos.scheduler.Call' message since not recovered yet
> [04:56:48]W: [Step 10/10] I1008 04:56:48.593413 25428 provisioner.cpp:253]
> Provisioner recovery complete
> [04:56:48]W: [Step 10/10] *** SIGABRT (@0x6334) received by PID 25396 (TID
> 0x7f1b02ed6700) from PID 25396; stack trace: ***
> [04:56:48]W: [Step 10/10] I1008 04:56:48.593520 25421
> containerizer.cpp:557] Recovering containerizer
> [04:56:48]W: [Step 10/10] I1008 04:56:48.593529 25425 slave.cpp:5276]
> Finished recovery
> [04:56:48]W: [Step 10/10] I1008 04:56:48.593627 25422 leveldb.cpp:304]
> Persisting metadata (8 bytes) to leveldb took 4.546422ms
> [04:56:48]W: [Step 10/10] I1008 04:56:48.593695 25428 provisioner.cpp:253]
> Provisioner recovery complete
> [04:56:48]W: [Step 10/10] I1008 04:56:48.593701 25422 replica.cpp:320]
> Persisted replica status to VOTING
> [04:56:48]W: [Step 10/10] I1008 04:56:48.593760 25424 slave.cpp:5276]
> Finished recovery
> [04:56:48]W: [Step 10/10] I1008 04:56:48.593864 25427 recover.cpp:582]
> Successfully joined the Paxos group
> [04:56:48]W: [Step 10/10] I1008 04:56:48.593896 25425 slave.cpp:5448]
> Querying resource estimator for oversubscribable resources
> [04:56:48]W: [Step 10/10] I1008 04:56:48.593922 25427 recover.cpp:466]
> Recover process terminated
> [04:56:48]W: [Step 10/10] I1008 04:56:48.593976 25427 slave.cpp:5462]
> Received oversubscribable resources {} from the resource estimator
> [04:56:48]W: [Step 10/10] I1008 04:56:48.594002 25424 slave.cpp:5448]
> Querying resource estimator for oversubscribable resources
> [04:56:48]W: [Step 10/10] I1008 04:56:48.594017 25422 log.cpp:553]
> Attempting to start the writer
> [04:56:48]W: [Step 10/10] I1008 04:56:48.594030 25428
> status_update_manager.cpp:177] Pausing sending status updates
> [04:56:48]W: [Step 10/10] I1008 04:56:48.594032 25427 slave.cpp:915] New
> master detected at [email protected]:45167
> [04:56:48]W: [Step 10/10] I1008 04:56:48.594055 25423 slave.cpp:915] New
> master detected at [email protected]:45167
> [04:56:48]W: [Step 10/10] I1008 04:56:48.594048 25428
> status_update_manager.cpp:177] Pausing sending status updates
> [04:56:48]W: [Step 10/10] I1008 04:56:48.594061 25427 slave.cpp:936] No
> credentials provided. Attempting to register without authentication
> [04:56:48]W: [Step 10/10] I1008 04:56:48.594106 25427 slave.cpp:947]
> Detecting new master
> [04:56:48]W: [Step 10/10] I1008 04:56:48.594071 25423 slave.cpp:936] No
> credentials provided. Attempting to register without authentication
> [04:56:48]W: [Step 10/10] @ 0x7f1b0bf4b340 (unknown)
> [04:56:48]W: [Step 10/10] I1008 04:56:48.594125 25423 slave.cpp:947]
> Detecting new master
> [04:56:48]W: [Step 10/10] I1008 04:56:48.594194 25423 slave.cpp:5462]
> Received oversubscribable resources {} from the resource estimator
> [04:56:48]W: [Step 10/10] I1008 04:56:48.594378 25422 replica.cpp:493]
> Replica received implicit promise request from
> __req_res__(3)@172.30.2.21:45167 with proposal 1
> [04:56:48]W: [Step 10/10] @ 0x7f1b0bbaccc9 (unknown)
> [04:56:48]W: [Step 10/10] @ 0x7f1b0bbb00d8 (unknown)
> [04:56:48]W: [Step 10/10] @ 0x7f1b0bbe9394 (unknown)
> [04:56:48]W: [Step 10/10] I1008 04:56:48.595368 25422 leveldb.cpp:304]
> Persisting metadata (8 bytes) to leveldb took 972334ns
> [04:56:48]W: [Step 10/10] I1008 04:56:48.595381 25422 replica.cpp:342]
> Persisted promised to 1
> [04:56:48]W: [Step 10/10] @ 0x7f1b0bbf566e (unknown)
> [04:56:48]W: [Step 10/10] @ 0x7f1b0d930925
> _ZNSt14_Function_base13_Base_managerIZN7process5deferIN5mesos8internal5slave5SlaveEiiSt12_PlaceholderILi1EES7_ILi2EEEENS1_9_DeferredIDTcl4bindadsrSt8functionIFvT0_T1_EEclcvSF__Efp1_fp2_EEEERKNS1_3PIDIT_EEMSJ_FvSC_SD_ET2_T3_EUliiE_E10_M_managerERSt9_Any_dataRKST_St18_Manager_operation
> [04:56:48]W: [Step 10/10] I1008 04:56:48.597909 25421 coordinator.cpp:238]
> Coordinator attempting to fill missing positions
> [04:56:48]W: [Step 10/10] I1008 04:56:48.598273 25423 replica.cpp:388]
> Replica received explicit promise request from
> __req_res__(4)@172.30.2.21:45167 for position 0 with proposal 2
> [04:56:48]W: [Step 10/10] @ 0x7f1b0d935b0b
> std::_Function_base::_Base_manager<>::_M_manager()
> [04:56:48]W: [Step 10/10] @ 0x7f1b0d8f2516
> os::internal::configureSignal()
> [04:56:48]W: [Step 10/10] I1008 04:56:48.599318 25423 leveldb.cpp:341]
> Persisting action (8 bytes) to leveldb took 1.024957ms
> [04:56:48]W: [Step 10/10] I1008 04:56:48.599333 25423 replica.cpp:708]
> Persisted action NOP at position 0
> [04:56:48]W: [Step 10/10] I1008 04:56:48.599630 25428 replica.cpp:537]
> Replica received write request for position 0 from
> __req_res__(5)@172.30.2.21:45167
> [04:56:48]W: [Step 10/10] I1008 04:56:48.599660 25428 leveldb.cpp:436]
> Reading position from leveldb took 16893ns
> [04:56:48]W: [Step 10/10] @ 0x7f1b0d904022
> mesos::internal::slave::Slave::initialize()
> [04:56:48]W: [Step 10/10] @ 0x7f1b0e0c6ed1
> process::ProcessManager::resume()
> [04:56:48]W: [Step 10/10] @ 0x7f1b0e0c7187
> _ZNSt6thread5_ImplISt12_Bind_simpleIFZN7process14ProcessManager12init_threadsEvEUt_vEEE6_M_runEv
> [04:56:48]W: [Step 10/10] @ 0x7f1b0c726a60 (unknown)
> [04:56:48]W: [Step 10/10] @ 0x7f1b0bf43182 start_thread
> [04:56:48]W: [Step 10/10] @ 0x7f1b0bc7047d (unknown)
> {code}
> This is seen specifically in Ubuntu 14.04
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
