I've seen a similar issue on my dev box. On Thursday, November 20, 2014, Vinod Kone <vinodk...@gmail.com> wrote:
> This particular test got blocked at the very beginning of the test. All it > did was start master, start slave, pause clock and wait for http request to > /metrics/snapshot to return. > > That last part does AWAIT_READY() with default duration of 10s. AFAICT, > that code (process::internal::await()) should return within 10s even if the > clock is paused. So it is strange that it didn't. > > My suspicion with recent libprocess changes is because 1) i haven't seen > this issue before and 2) seems to be related to the guts of > libprocess/clock/timers that were recently touched. Of course, it could be > totally unrelated to libprocess changes too. > > On Thu, Nov 20, 2014 at 5:34 PM, Benjamin Hindman <b...@mesosphere.io > <javascript:;>> > wrote: > > > What led you to that? > > > > On Thu, Nov 20, 2014 at 4:19 PM, Vinod Kone <vinodk...@gmail.com > <javascript:;>> wrote: > > > >> +benh, joris > >> > >> from the looks of it, i suspect the test is doing a busy loop in > >> > https://github.com/apache/mesos/blob/master/3rdparty/libprocess/include/process/gtest.hpp#L48 > >> > >> could this be related to recent libprocess changes? > >> > >> On Thu, Nov 20, 2014 at 4:07 PM, Vinod Kone <vinodk...@gmail.com > <javascript:;>> wrote: > >> > >>> too bad there is no gdb trace on this box to understand where this test > >>> is stuck :( > >>> > >>> On Thu, Nov 20, 2014 at 3:55 PM, Apache Jenkins Server < > >>> jenk...@builds.apache.org <javascript:;>> wrote: > >>> > >>>> See < > >>>> > https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Disable-Java-Disable-Python-Disable-Webui/2581/changes > >>>> > > >>>> > >>>> Changes: > >>>> > >>>> [niklas] Changed master label test to strictly enforce format. > >>>> > >>>> [niklas] Changed slave label test to strictly enforce format. > >>>> > >>>> [tnachen] Match future dispatch messages with type info. > >>>> > >>>> [tnachen] Fixed future dispatch in tests to match function type. > >>>> > >>>> ------------------------------------------ > >>>> [...truncated 3727 lines...] > >>>> make[3]: `no-executor-framework' is up to date. > >>>> make[3]: `docker-no-executor-framework' is up to date. > >>>> make[3]: `balloon-framework' is up to date. > >>>> make[3]: `balloon-executor' is up to date. > >>>> make[3]: `load-generator-framework' is up to date. > >>>> make[3]: `setns-test-helper' is up to date. > >>>> make[3]: `mesos-tests' is up to date. > >>>> make[3]: Nothing to be done for > >>>> `../../src/tests/balloon_framework_test.sh'. > >>>> make[3]: Nothing to be done for > >>>> `../../src/tests/low_level_scheduler_libprocess_test.sh'. > >>>> make[3]: Nothing to be done for > >>>> `../../src/tests/low_level_scheduler_pthread_test.sh'. > >>>> make[3]: Nothing to be done for > >>>> `../../src/tests/java_exception_test.sh'. > >>>> make[3]: Nothing to be done for > >>>> `../../src/tests/java_framework_test.sh'. > >>>> make[3]: Nothing to be done for `../../src/tests/java_log_test.sh'. > >>>> make[3]: Nothing to be done for > >>>> `../../src/tests/no_executor_framework_test.sh'. > >>>> make[3]: Nothing to be done for > >>>> `../../src/tests/python_framework_test.sh'. > >>>> make[3]: Nothing to be done for > >>>> `../../src/tests/test_framework_test.sh'. > >>>> make[3]: Leaving directory `< > >>>> > https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Disable-Java-Disable-Python-Disable-Webui/ws/build/src > >>>> '> > >>>> make check-local > >>>> make[3]: Entering directory `< > >>>> > https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Disable-Java-Disable-Python-Disable-Webui/ws/build/src > >>>> '> > >>>> ./mesos-tests > >>>> WARNING: Logging before InitGoogleLogging() is written to STDERR > >>>> I1120 23:24:12.553694 30655 process.cpp:983] libprocess is initialized > >>>> on 67.195.81.186:33912 for 16 cpus > >>>> I1120 23:24:12.553962 30655 logging.cpp:177] Logging to STDERR > >>>> Source directory: < > >>>> > https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Disable-Java-Disable-Python-Disable-Webui/ws/ > >>>> > > >>>> Build directory: < > >>>> > https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Disable-Java-Disable-Python-Disable-Webui/ws/build > >>>> > > >>>> ------------------------------------------------------------- > >>>> We cannot run any cgroups tests that require mounting > >>>> hierarchies because you have the following hierarchies mounted: > >>>> /sys/fs/cgroup/blkio, /sys/fs/cgroup/cpu, /sys/fs/cgroup/cpuacct, > >>>> /sys/fs/cgroup/cpuset, /sys/fs/cgroup/devices, /sys/fs/cgroup/freezer, > >>>> /sys/fs/cgroup/hugetlb, /sys/fs/cgroup/memory, > /sys/fs/cgroup/perf_event, > >>>> /sys/fs/cgroup/systemd > >>>> We'll disable the CgroupsNoHierarchyTest test fixture for now. > >>>> ------------------------------------------------------------- > >>>> Note: Google Test filter = > >>>> > *-DockerContainerizerTest.ROOT_DOCKER_Launch_Executor:DockerContainerizerTest.ROOT_DOCKER_Launch_Executor_Bridged:DockerContainerizerTest.ROOT_DOCKER_Launch:DockerContainerizerTest.ROOT_DOCKER_Kill:DockerContainerizerTest.ROOT_DOCKER_Usage:DockerContainerizerTest.ROOT_DOCKER_Update:DockerContainerizerTest.DISABLED_ROOT_DOCKER_Recover:DockerContainerizerTest.ROOT_DOCKER_Logs:DockerContainerizerTest.ROOT_DOCKER_Default_CMD:DockerContainerizerTest.ROOT_DOCKER_Default_CMD_Override:DockerContainerizerTest.ROOT_DOCKER_Default_CMD_Args:DockerContainerizerTest.ROOT_DOCKER_SlaveRecoveryTaskContainer:DockerContainerizerTest.DISABLED_ROOT_DOCKER_SlaveRecoveryExecutorContainer:DockerContainerizerTest.ROOT_DOCKER_PortMapping:DockerContainerizerTest.ROOT_DOCKER_LaunchSandboxWithColon:DockerContainerizerTest.ROOT_DOCKER_DestroyWhileFetching:DockerContainerizerTest.ROOT_DOCKER_DestroyWhilePulling:DockerTest.ROOT_DOCKER_interface:DockerTest.ROOT_DOCKER_CheckCommandWithShell:DockerTest.ROOT_DOCKER_CheckPortResource:DockerTest.ROOT_DOCKER_CancelPull:CpuIsolatorTest/1.UserCpuUsage:CpuIsolatorTest/1.SystemCpuUsage:LimitedCpuIsolatorTest.ROOT_CGROUPS_Cfs:LimitedCpuIsolatorTest.ROOT_CGROUPS_Cfs_Big_Quota:MemIsolatorTest/0.MemUsage:MemIsolatorTest/1.MemUsage:MemIsolatorTest/2.MemUsage:PerfEventIsolatorTest.ROOT_CGROUPS_Sample:SharedFilesystemIsolatorTest.ROOT_RelativeVolume:SharedFilesystemIsolatorTest.ROOT_AbsoluteVolume:NamespacesPidIsolatorTest.ROOT_PidNamespace:UserCgroupIsolatorTest/0.ROOT_CGROUPS_UserCgroup:UserCgroupIsolatorTest/1.ROOT_CGROUPS_UserCgroup:UserCgroupIsolatorTest/2.ROOT_CGROUPS_UserCgroup:MesosContainerizerSlaveRecoveryTest.CGROUPS_ROOT_PerfRollForward:MesosContainerizerSlaveRecoveryTest.CGROUPS_ROOT_PidNamespaceForward:MesosContainerizerSlaveRecoveryTest.CGROUPS_ROOT_PidNamespaceBackward:SlaveTest.ROOT_RunTaskWithCommandInfoWithoutUser:SlaveTest.DISABLED_ROOT_RunTaskWithCommandInfoWithUser:ContainerizerTest.ROOT_CGROUPS_BalloonFramework:CgroupsAnyHierarchyTest.ROOT_CGROUPS_Enabled:CgroupsAnyHierarchyTest.ROOT_CGROUPS_Subsystems:CgroupsAnyHierarchyTest.ROOT_CGROUPS_Mounted:CgroupsAnyHierarchyTest.ROOT_CGROUPS_Get:CgroupsAnyHierarchyTest.ROOT_CGROUPS_NestedCgroups:CgroupsAnyHierarchyTest.ROOT_CGROUPS_Tasks:CgroupsAnyHierarchyTest.ROOT_CGROUPS_Read:CgroupsAnyHierarchyTest.ROOT_CGROUPS_Write:CgroupsAnyHierarchyTest.ROOT_CGROUPS_Cfs_Big_Quota:CgroupsAnyHierarchyWithCpuMemoryTest.ROOT_CGROUPS_Busy:CgroupsAnyHierarchyWithCpuMemoryTest.ROOT_CGROUPS_SubsystemsHierarchy:CgroupsAnyHierarchyWithCpuMemoryTest.ROOT_CGROUPS_FindCgroupSubsystems:CgroupsAnyHierarchyWithCpuMemoryTest.ROOT_CGROUPS_MountedSubsystems:CgroupsAnyHierarchyWithCpuMemoryTest.ROOT_CGROUPS_CreateRemove:CgroupsAnyHierarchyWithCpuMemoryTest.ROOT_CGROUPS_Listen:CgroupsAnyHierarchyWithCpuMemoryTest.ROOT_CGROUPS_FreezeNonFreezer:CgroupsNoHierarchyTest.ROOT_CGROUPS_NOHIERARCHY_MountUnmountHierarchy:CgroupsAnyHierarchyWithCpuAcctMemoryTest.ROOT_CGROUPS_Stat:CgroupsAnyHierarchyWithFreezerTest.ROOT_CGROUPS_Freeze:CgroupsAnyHierarchyWithFreezerTest.ROOT_CGROUPS_Kill:CgroupsAnyHierarchyWithFreezerTest.ROOT_CGROUPS_Destroy:CgroupsAnyHierarchyWithFreezerTest.ROOT_CGROUPS_AssignThreads:CgroupsAnyHierarchyWithFreezerTest.ROOT_CGROUPS_DestroyStoppedProcess:CgroupsAnyHierarchyWithFreezerTest.ROOT_CGROUPS_DestroyTracedProcess:CgroupsAnyHierarchyWithPerfEventTest.ROOT_CGROUPS_Perf:NsTest.ROOT_setns:NsTest.ROOT_setnsMultipleThreads:NsTest.ROOT_getns:NsTest.ROOT_destroy:PerfTest.ROOT_Events:PerfTest.ROOT_SampleInit:SlaveCount/Registrar_BENCHMARK_Test.performance/0:SlaveCount/Registrar_BENCHMARK_Test.performance/1:SlaveCount/Registrar_BENCHMARK_Test.performance/2:SlaveCount/Registrar_BENCHMARK_Test.performance/3 > >>>> Note: Randomizing tests' orders with a seed of 47110 . > >>>> [==========] Running 437 tests from 68 test cases. > >>>> [----------] Global test environment set-up. > >>>> [----------] 4 tests from CRAMMD5Authentication/2, where TypeParam = > >>>> > mesos::internal::cram_md5::Authentication<mesos::internal::cram_md5::CRAMMD5Authenticatee, > >>>> mesos::internal::tests::Module<mesos::internal::Authenticator, > >>>> (mesos::internal::tests::ModuleID)3> > > >>>> [ RUN ] CRAMMD5Authentication/2.failed1 > >>>> Using temporary directory > '/tmp/CRAMMD5Authentication_2_failed1_V0pTyx' > >>>> I1120 23:24:12.685442 30676 authenticatee.hpp:114] Initializing client > >>>> SASL > >>>> I1120 23:24:12.686884 30676 authenticatee.hpp:138] Creating new client > >>>> SASL connection > >>>> I1120 23:24:12.687741 30678 authenticator.hpp:108] Initializing server > >>>> SASL > >>>> I1120 23:24:12.687944 30678 auxprop.cpp:63] Initialized in-memory > >>>> auxiliary property plugin > >>>> I1120 23:24:12.687966 30678 authenticator.hpp:170] Creating new server > >>>> SASL connection > >>>> I1120 23:24:12.688140 30678 authenticatee.hpp:229] Received SASL > >>>> authentication mechanisms: CRAM-MD5 > >>>> I1120 23:24:12.688180 30678 authenticatee.hpp:255] Attempting to > >>>> authenticate with mechanism 'CRAM-MD5' > >>>> I1120 23:24:12.688294 30678 authenticator.hpp:276] Received SASL > >>>> authentication start > >>>> I1120 23:24:12.688361 30678 authenticator.hpp:398] Authentication > >>>> requires more steps > >>>> I1120 23:24:12.688472 30673 authenticatee.hpp:275] Received SASL > >>>> authentication step > >>>> I1120 23:24:12.688617 30673 authenticator.hpp:304] Received SASL > >>>> authentication step > >>>> I1120 23:24:12.688649 30673 auxprop.cpp:99] Request to lookup > >>>> properties for user: 'benh' realm: 'penates.apache.org' server FQDN: > ' > >>>> penates.apache.org' SASL_AUXPROP_VERIFY_AGAINST_HASH: false > >>>> SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false > >>>> I1120 23:24:12.688663 30673 auxprop.cpp:171] Looking up auxiliary > >>>> property '*userPassword' > >>>> I1120 23:24:12.688814 30673 auxprop.cpp:171] Looking up auxiliary > >>>> property '*cmusaslsecretCRAM-MD5' > >>>> I1120 23:24:12.688846 30673 auxprop.cpp:99] Request to lookup > >>>> properties for user: 'benh' realm: 'penates.apache.org' server FQDN: > ' > >>>> penates.apache.org' SASL_AUXPROP_VERIFY_AGAINST_HASH: false > >>>> SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true > >>>> I1120 23:24:12.688858 30673 auxprop.cpp:121] Skipping auxiliary > >>>> property '*userPassword' since SASL_AUXPROP_AUTHZID == true > >>>> I1120 23:24:12.688863 30673 auxprop.cpp:121] Skipping auxiliary > >>>> property '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true > >>>> W1120 23:24:12.688887 30673 authenticator.hpp:404] Authentication > >>>> failure: authentication failure > >>>> [ OK ] CRAMMD5Authentication/2.failed1 (11 ms) > >>>> [ RUN ] CRAMMD5Authentication/2.AuthenticatorDestructionRace > >>>> Using temporary directory > >>>> '/tmp/CRAMMD5Authentication_2_AuthenticatorDestructionRace_CRclP9' > >>>> I1120 23:24:12.690254 30684 authenticatee.hpp:138] Creating new client > >>>> SASL connection > >>>> I1120 23:24:12.690830 30676 authenticator.hpp:170] Creating new server > >>>> SASL connection > >>>> I1120 23:24:12.690989 30682 authenticatee.hpp:229] Received SASL > >>>> authentication mechanisms: CRAM-MD5 > >>>> I1120 23:24:12.691015 30682 authenticatee.hpp:255] Attempting to > >>>> authenticate with mechanism 'CRAM-MD5' > >>>> I1120 23:24:12.691097 30671 authenticator.hpp:276] Received SASL > >>>> authentication start > >>>> I1120 23:24:12.691135 30671 authenticator.hpp:398] Authentication > >>>> requires more steps > >>>> [ OK ] CRAMMD5Authentication/2.AuthenticatorDestructionRace (2 > ms) > >>>> [ RUN ] CRAMMD5Authentication/2.success > >>>> Using temporary directory > '/tmp/CRAMMD5Authentication_2_success_JZF85L' > >>>> I1120 23:24:12.692466 30682 authenticatee.hpp:138] Creating new client > >>>> SASL connection > >>>> I1120 23:24:12.692939 30676 authenticator.hpp:170] Creating new server > >>>> SASL connection > >>>> I1120 23:24:12.693069 30684 authenticatee.hpp:229] Received SASL > >>>> authentication mechanisms: CRAM-MD5 > >>>> I1120 23:24:12.693094 30684 authenticatee.hpp:255] Attempting to > >>>> authenticate with mechanism 'CRAM-MD5' > >>>> I1120 23:24:12.693171 30682 authenticator.hpp:276] Received SASL > >>>> authentication start > >>>> I1120 23:24:12.693207 30682 authenticator.hpp:398] Authentication > >>>> requires more steps > >>>> I1120 23:24:12.693274 30684 authenticatee.hpp:275] Received SASL > >>>> authentication step > >>>> I1120 23:24:12.693366 30682 authenticator.hpp:304] Received SASL > >>>> authentication step > >>>> I1120 23:24:12.693394 30682 auxprop.cpp:99] Request to lookup > >>>> properties for user: 'benh' realm: 'penates.apache.org' server FQDN: > ' > >>>> penates.apache.org' SASL_AUXPROP_VERIFY_AGAINST_HASH: false > >>>> SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false > >>>> I1120 23:24:12.693408 30682 auxprop.cpp:171] Looking up auxiliary > >>>> property '*userPassword' > >>>> I1120 23:24:12.693431 30682 auxprop.cpp:171] Looking up auxiliary > >>>> property '*cmusaslsecretCRAM-MD5' > >>>> I1120 23:24:12.693449 30682 auxprop.cpp:99] Request to lookup > >>>> properties for user: 'benh' realm: 'penates.apache.org' server FQDN: > ' > >>>> penates.apache.org' SASL_AUXPROP_VERIFY_AGAINST_HASH: false > >>>> SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true > >>>> I1120 23:24:12.693456 30682 auxprop.cpp:121] Skipping auxiliary > >>>> property '*userPassword' since SASL_AUXPROP_AUTHZID == true > >>>> I1120 23:24:12.693461 30682 auxprop.cpp:121] Skipping auxiliary > >>>> property '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true > >>>> I1120 23:24:12.693476 30682 authenticator.hpp:390] Authentication > >>>> success > >>>> I1120 23:24:12.693544 30684 authenticatee.hpp:315] Authentication > >>>> success > >>>> [ OK ] CRAMMD5Authentication/2.success (3 ms) > >>>> [ RUN ] CRAMMD5Authentication/2.failed2 > >>>> Using temporary directory > '/tmp/CRAMMD5Authentication_2_failed2_yugjno' > >>>> I1120 23:24:12.694799 30677 authenticatee.hpp:138] Creating new client > >>>> SASL connection > >>>> I1120 23:24:12.695314 30676 authenticator.hpp:170] Creating new server > >>>> SASL connection > >>>> I1120 23:24:12.695435 30674 authenticatee.hpp:229] Received SASL > >>>> authentication mechanisms: CRAM-MD5 > >>>> I1120 23:24:12.695457 30674 authenticatee.hpp:255] Attempting to > >>>> authenticate with mechanism 'CRAM-MD5' > >>>> I1120 23:24:12.695544 30670 authenticator.hpp:276] Received SASL > >>>> authentication start > >>>> I1120 23:24:12.695582 30670 authenticator.hpp:398] Authentication > >>>> requires more steps > >>>> I1120 23:24:12.695657 30674 authenticatee.hpp:275] Received SASL > >>>> authentication step > >>>> I1120 23:24:12.695745 30674 authenticator.hpp:304] Received SASL > >>>> authentication step > >>>> I1120 23:24:12.695770 30674 auxprop.cpp:99] Request to lookup > >>>> properties for user: 'benh' realm: 'penates.apache.org' server FQDN: > ' > >>>> penates.apache.org' SASL_AUXPROP_VERIFY_AGAINST_HASH: false > >>>> SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false > >>>> I1120 23:24:12.695778 30674 auxprop.cpp:171] Looking up auxiliary > >>>> property '*userPassword' > >>>> I1120 23:24:12.695788 30674 auxprop.cpp:171] Looking up auxiliary > >>>> property '*cmusaslsecretCRAM-MD5' > >>>> I1120 23:24:12.695797 30674 auxprop.cpp:99] Request to lookup > >>>> properties for user: 'benh' realm: 'penates.apache.org' server FQDN: > ' > >>>> penates.apache.org' SASL_AUXPROP_VERIFY_AGAINST_HASH: false > >>>> SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true > >>>> I1120 23:24:12.695803 30674 auxprop.cpp:121] Skipping auxiliary > >>>> property '*userPassword' since SASL_AUXPROP_AUTHZID == true > >>>> I1120 23:24:12.695809 30674 auxprop.cpp:121] Skipping auxiliary > >>>> property '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true > >>>> W1120 23:24:12.695818 30674 authenticator.hpp:404] Authentication > >>>> failure: user not found > >>>> [ OK ] CRAMMD5Authentication/2.failed2 (2 ms) > >>>> [----------] 4 tests from CRAMMD5Authentication/2 (18 ms total) > >>>> > >>>> [----------] 6 tests from RateLimitingTest > >>>> [ RUN ] RateLimitingTest.NoRateLimiting > >>>> Using temporary directory > '/tmp/RateLimitingTest_NoRateLimiting_XteRE0' > >>>> I1120 23:24:12.701375 30655 leveldb.cpp:176] Opened db in 3.4075ms > >>>> I1120 23:24:12.703060 30655 leveldb.cpp:183] Compacted db in 1.64536ms > >>>> I1120 23:24:12.703178 30655 leveldb.cpp:198] Created db iterator in > >>>> 78140ns > >>>> I1120 23:24:12.703205 30655 leveldb.cpp:204] Seeked to beginning of db > >>>> in 3649ns > >>>> I1120 23:24:12.703217 30655 leveldb.cpp:273] Iterated through 0 keys > in > >>>> the db in 400ns > >>>> I1120 23:24:12.703368 30655 replica.cpp:741] Replica recovered with > log > >>>> positions 0 -> 0 with 1 holes and 0 unlearned > >>>> I1120 23:24:12.705715 30678 recover.cpp:437] Starting replica recovery > >>>> I1120 23:24:12.706840 30678 recover.cpp:463] Replica is in EMPTY > status > >>>> I1120 23:24:12.711232 30681 replica.cpp:638] Replica in EMPTY status > >>>> received a broadcasted recover request > >>>> I1120 23:24:12.712901 30683 recover.cpp:188] Received a recover > >>>> response from a replica in EMPTY status > >>>> I1120 23:24:12.713747 30673 recover.cpp:554] Updating replica status > to > >>>> STARTING > >>>> I1120 23:24:12.714715 30683 leveldb.cpp:306] Persisting metadata (8 > >>>> bytes) to leveldb took 708621ns > >>>> I1120 23:24:12.714754 30683 replica.cpp:320] Persisted replica status > >>>> to STARTING > >>>> I1120 23:24:12.715100 30684 recover.cpp:463] Replica is in STARTING > >>>> status > >>>> I1120 23:24:12.716419 30679 replica.cpp:638] Replica in STARTING > status > >>>> received a broadcasted recover request > >>>> I1120 23:24:12.716961 30676 recover.cpp:188] Received a recover > >>>> response from a replica in STARTING status > >>>> I1120 23:24:12.717490 30677 recover.cpp:554] Updating replica status > to > >>>> VOTING > >>>> I1120 23:24:12.718242 30680 leveldb.cpp:306] Persisting metadata (8 > >>>> bytes) to leveldb took 484920ns > >>>> I1120 23:24:12.718276 30680 replica.cpp:320] Persisted replica status > >>>> to VOTING > >>>> I1120 23:24:12.718385 30672 recover.cpp:568] Successfully joined the > >>>> Paxos group > >>>> I1120 23:24:12.718698 30672 recover.cpp:452] Recover process > terminated > >>>> I1120 23:24:12.723183 30670 master.cpp:318] Master > >>>> 20141120-232412-3125920579-33912-30655 (penates.apache.org) started > on > >>>> 67.195.81.186:33912 > >>>> I1120 23:24:12.723278 30670 master.cpp:364] Master only allowing > >>>> authenticated frameworks to register > >>>> I1120 23:24:12.723353 30670 master.cpp:369] Master only allowing > >>>> authenticated slaves to register > >>>> I1120 23:24:12.723420 30670 credentials.hpp:36] Loading credentials > for > >>>> authentication from > >>>> '/tmp/RateLimitingTest_NoRateLimiting_XteRE0/credentials' > >>>> I1120 23:24:12.723983 30670 master.cpp:413] Authorization enabled > >>>> I1120 23:24:12.724187 30670 master.cpp:460] Framework rate limiting > >>>> enabled > >>>> I1120 23:24:12.724884 30678 master.cpp:126] No whitelist given. > >>>> Advertising offers for all slaves > >>>> I1120 23:24:12.725131 30684 hierarchical_allocator_process.hpp:299] > >>>> Initializing hierarchical allocator process with master : > >>>> master@67.195.81.186:33912 > >>>> I1120 23:24:12.728044 30676 master.cpp:1263] The newly elected leader > >>>> is master@67.195.81.186:33912 with id > >>>> 20141120-232412-3125920579-33912-30655 > >>>> I1120 23:24:12.728078 30676 master.cpp:1276] Elected as the leading > >>>> master! > >>>> I1120 23:24:12.728116 30676 master.cpp:1094] Recovering from registrar > >>>> I1120 23:24:12.728454 30679 registrar.cpp:313] Recovering registrar > >>>> I1120 23:24:12.730170 30677 log.cpp:656] Attempting to start the > writer > >>>> I1120 23:24:12.733129 30676 replica.cpp:474] Replica received implicit > >>>> promise request with proposal 1 > >>>> I1120 23:24:12.733763 30676 leveldb.cpp:306] Persisting metadata (8 > >>>> bytes) to leveldb took 598467ns > >>>> I1120 23:24:12.733795 30676 replica.cpp:342] Persisted promised to 1 > >>>> I1120 23:24:12.735090 30680 coordinator.cpp:230] Coordinator attemping > >>>> to fill missing position > >>>> I1120 23:24:12.737241 30677 replica.cpp:375] Replica received explicit > >>>> promise request for position 0 with proposal 2 > >>>> I1120 23:24:12.737690 30677 leveldb.cpp:343] Persisting action (8 > >>>> bytes) to leveldb took 402621ns > >>>> I1120 23:24:12.737721 30677 replica.cpp:676] Persisted action at 0 > >>>> I1120 23:24:12.739233 30681 replica.cpp:508] Replica received write > >>>> request for position 0 > >>>> I1120 23:24:12.739351 30681 leveldb.cpp:438] Reading position from > >>>> leveldb took 79235ns > >>>> I1120 23:24:12.739878 30681 leveldb.cpp:343] Persisting action (14 > >>>> bytes) to leveldb took 448572ns > >>>> I1120 23:24:12.739907 30681 replica.cpp:676] Persisted action at 0 > >>>> I1120 23:24:12.740696 30679 replica.cpp:655] Replica received learned > >>>> notice for position 0 > >>>> I1120 23:24:12.741052 30679 leveldb.cpp:343] Persisting action (16 > >>>> bytes) to leveldb took 322694ns > >>>> I1120 23:24:12.741083 30679 replica.cpp:676] Persisted action at 0 > >>>> I1120 23:24:12.741117 30679 replica.cpp:661] Replica learned NOP > action > >>>> at position 0 > >>>> I1120 23:24:12.741811 30672 log.cpp:672] Writer started with ending > >>>> position 0 > >>>> I1120 23:24:12.744951 30670 leveldb.cpp:438] Reading position from > >>>> leveldb took 30629ns > >>>> I1120 23:24:12.748874 30678 registrar.cpp:346] Successfully fetched > the > >>>> registry (0B) in 20.35584ms > >>>> I1120 23:24:12.749171 30678 registrar.cpp:445] Applied 1 operations in > >>>> 87917ns; attempting to update the 'registry' > >>>> I1120 23:24:12.752719 30672 log.cpp:680] Attempting to append 140 > bytes > >>>> to the log > >>>> I1120 23:24:12.752936 30671 coordinator.cpp:340] Coordinator > attempting > >>>> to write APPEND action at position 1 > >>>> I1120 23:24:12.753829 30671 replica.cpp:508] Replica received write > >>>> request for position 1 > >>>> I1120 23:24:12.754359 30671 leveldb.cpp:343] Persisting action (159 > >>>> bytes) to leveldb took 494376ns > >>>> I1120 23:24:12.754387 30671 replica.cpp:676] Persisted action at 1 > >>>> I1120 23:24:12.755213 30677 replica.cpp:655] Replica received learned > >>>> notice for position 1 > >>>> I1120 23:24:12.755621 30677 leveldb.cpp:343] Persisting action (161 > >>>> bytes) to leveldb took 375152ns > >>>> I1120 23:24:12.755648 30677 replica.cpp:676] Persisted action at 1 > >>>> I1120 23:24:12.755669 30677 replica.cpp:661] Replica learned APPEND > >>>> action at position 1 > >>>> I1120 23:24:12.757231 30673 registrar.cpp:490] Successfully updated > the > >>>> 'registry' in 7.922944ms > >>>> I1120 23:24:12.757463 30673 registrar.cpp:376] Successfully recovered > >>>> registrar > >>>> I1120 23:24:12.757665 30682 log.cpp:699] Attempting to truncate the > log > >>>> to 1 > >>>> I1120 23:24:12.757859 30679 coordinator.cpp:340] Coordinator > attempting > >>>> to write TRUNCATE action at position 2 > >>>> I1120 23:24:12.757899 30681 master.cpp:1121] Recovered 0 slaves from > >>>> the Registry (102B) ; allowing 10mins for slaves to re-register > >>>> I1120 23:24:12.758951 30680 replica.cpp:508] Replica received write > >>>> request for position 2 > >>>> I1120 23:24:12.759395 30680 leveldb.cpp:343] Persisting action (16 > >>>> bytes) to leveldb took 408946ns > >>>> I1120 23:24:12.759429 30680 replica.cpp:676] Persisted action at 2 > >>>> I1120 23:24:12.760109 30675 replica.cpp:655] Replica received learned > >>>> notice for position 2 > >>>> I1120 23:24:12.760496 30675 leveldb.cpp:343] Persisting action (18 > >>>> bytes) to leveldb took 356913ns > >>>> I1120 23:24:12.760584 30675 leveldb.cpp:401] Deleting ~1 keys from > >>>> leveldb took 45969ns > >>>> I1120 23:24:12.760608 30675 replica.cpp:676] Persisted action at 2 > >>>> I1120 23:24:12.760633 30675 replica.cpp:661] Replica learned TRUNCATE > >>>> action at position 2 > >>>> I1120 23:24:12.779798 30680 process.cpp:2881] Handling HTTP event for > >>>> process 'metrics' with path: '/metrics/snapshot' > >>>> Thu Nov 20 23:54:06 UTC 2014: process still running after 1800 seconds > >>>> ========== > >>>> > >>>> Attaching gdb to 30437 bash ../support/timed_tests.sh GLOG_v=1 > >>>> MESOS_VERBOSE=1 make check GTEST_SHUFFLE=1 GTEST_OUTPUT=xml: 1800 > >>>> > >>>> ========== > >>>> ../support/timed_tests.sh: line 72: gdb: command not found > >>>> ========== > >>>> > >>>> Attaching gdb to 30440 bash ../support/timed_tests.sh GLOG_v=1 > >>>> MESOS_VERBOSE=1 make check GTEST_SHUFFLE=1 GTEST_OUTPUT=xml: 1800 > >>>> > >>>> ========== > >>>> ../support/timed_tests.sh: line 72: gdb: command not found > >>>> ========== > >>>> > >>>> Attaching gdb to 30442 make check GTEST_SHUFFLE=1 GTEST_OUTPUT=xml: > >>>> > >>>> ========== > >>>> ../support/timed_tests.sh: line 72: gdb: command not found > >>>> ========== > >>>> > >>>> Attaching gdb to 30446 /bin/bash -c fail=; \ if (target_option=k; case > >>>> ${target_option-} in ?) ;; *) echo "am__make_running_with_option: > internal > >>>> error: invalid" "target option '${target_option-}' specified" >&2; > exit 1;; > >>>> esac; has_opt=no; sane_makeflags=$MAKEFLAGS; if test -n ' Makefile' > && test > >>>> -n '0'; then sane_makeflags=$MFLAGS; else case $MAKEFLAGS in *\\[\ > \.]*) > >>>> bs=\\; sane_makeflags=`printf '%s\n' "$MAKEFLAGS" | sed "s/$bs$bs[$bs > >>>> $bs.]*//g"`;; esac; fi; skip_next=no; strip_trailopt () { flg=`printf > >>>> '%s\n' "$flg" | sed "s/$1.*$//"`; }; for flg in $sane_makeflags; do > test > >>>> $skip_next = yes && { skip_next=no; continue; }; case $flg in *=*|--*) > >>>> continue;; -*I) strip_trailopt 'I'; skip_next=yes;; -*I?*) > strip_trailopt > >>>> 'I';; -*O) strip_trailopt 'O'; skip_next=yes;; -*O?*) strip_trailopt > 'O';; > >>>> -*l) strip_trailopt 'l'; skip_next=yes;; -*l?*) strip_trailopt 'l';; > >>>> -[dEDm]) skip_next=yes;; -[JT]) skip_next=yes;; esac; case $flg in > >>>> *$target_option*) has_opt=yes; break;; esac; done; test $has_opt = > yes); > >>>> then \ failcom='fail=yes'; \ else \ failcom='exit 1'; \ fi; \ > >>>> dot_seen=no; \ target=`echo check-recursive | sed s/-recursive//`; \ > case > >>>> "check-recursive" in \ distclean-* | maintainer-clean-*) list='. > 3rdparty > >>>> src ec2' ;; \ *) list='. 3rdparty src ec2' ;; \ esac; \ for subdir > in > >>>> $list; do \ echo "Making $target in $subdir"; \ if test "$subdir" > = > >>>> "."; then \ dot_seen=yes; \ local_target="$target-am"; \ > else \ > >>>> local_target="$target"; \ fi; \ (CDPATH="${ZSH_VERSION+.}:" && > cd > >>>> $subdir && make $local_target) \ || eval $failcom; \ done; \ if > test > >>>> "$dot_seen" = "no"; then \ make "$target-am" || exit 1; \ fi; test > -z > >>>> "$fail" > >>>> > >>>> ========== > >>>> ../support/timed_tests.sh: line 72: gdb: command not found > >>>> ========== > >>>> > >>>> Attaching gdb to 30642 /bin/bash -c fail=; \ if (target_option=k; case > >>>> ${target_option-} in ?) ;; *) echo "am__make_running_with_option: > internal > >>>> error: invalid" "target option '${target_option-}' specified" >&2; > exit 1;; > >>>> esac; has_opt=no; sane_makeflags=$MAKEFLAGS; if test -n ' Makefile' > && test > >>>> -n '0'; then sane_makeflags=$MFLAGS; else case $MAKEFLAGS in *\\[\ > \.]*) > >>>> bs=\\; sane_makeflags=`printf '%s\n' "$MAKEFLAGS" | sed "s/$bs$bs[$bs > >>>> $bs.]*//g"`;; esac; fi; skip_next=no; strip_trailopt () { flg=`printf > >>>> '%s\n' "$flg" | sed "s/$1.*$//"`; }; for flg in $sane_makeflags; do > test > >>>> $skip_next = yes && { skip_next=no; continue; }; case $flg in *=*|--*) > >>>> continue;; -*I) strip_trailopt 'I'; skip_next=yes;; -*I?*) > strip_trailopt > >>>> 'I';; -*O) strip_trailopt 'O'; skip_next=yes;; -*O?*) strip_trailopt > 'O';; > >>>> -*l) strip_trailopt 'l'; skip_next=yes;; -*l?*) strip_trailopt 'l';; > >>>> -[dEDm]) skip_next=yes;; -[JT]) skip_next=yes;; esac; case $flg in > >>>> *$target_option*) has_opt=yes; break;; esac; done; test $has_opt = > yes); > >>>> then \ failcom='fail=yes'; \ else \ failcom='exit 1'; \ fi; \ > >>>> dot_seen=no; \ target=`echo check-recursive | sed s/-recursive//`; \ > case > >>>> "check-recursive" in \ distclean-* | maintainer-clean-*) list='. > 3rdparty > >>>> src ec2' ;; \ *) list='. 3rdparty src ec2' ;; \ esac; \ for subdir > in > >>>> $list; do \ echo "Making $target in $subdir"; \ if test "$subdir" > = > >>>> "."; then \ dot_seen=yes; \ local_target="$target-am"; \ > else \ > >>>> local_target="$target"; \ fi; \ (CDPATH="${ZSH_VERSION+.}:" && > cd > >>>> $subdir && make $local_target) \ || eval $failcom; \ done; \ if > test > >>>> "$dot_seen" = "no"; then \ make "$target-am" || exit 1; \ fi; test > -z > >>>> "$fail" > >>>> > >>>> ========== > >>>> ../support/timed_tests.sh: line 72: gdb: command not found > >>>> ========== > >>>> > >>>> Attaching gdb to 30643 make check > >>>> > >>>> ========== > >>>> ../support/timed_tests.sh: line 72: gdb: command not found > >>>> ========== > >>>> > >>>> Attaching gdb to 30645 make check-am > >>>> > >>>> ========== > >>>> ../support/timed_tests.sh: line 72: gdb: command not found > >>>> ========== > >>>> > >>>> Attaching gdb to 30652 make check-local > >>>> > >>>> ========== > >>>> ../support/timed_tests.sh: line 72: gdb: command not found > >>>> ========== > >>>> > >>>> Attaching gdb to 30655 < > >>>> > https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Disable-Java-Disable-Python-Disable-Webui/ws/build/src/.libs/lt-mesos-tests > >>>> > > >>>> > >>>> ========== > >>>> ../support/timed_tests.sh: line 72: gdb: command not found > >>>> Test failed and killing the stuck test process > >>>> W1120 23:24:12.779798 30669 logging.cpp:81] RAW: Received signal > >>>> SIGTERM from process 30652 of user 2396; exiting > >>>> W1120 23:24:12.779798 30655 logging.cpp:81] RAW: Received signal > >>>> SIGTERM from process 20380 of user 2396; exiting > >>>> make: *** [check-recursive] Terminated > >>>> /tmp/hudson9138584248396286316.sh: line 16: 30437 Terminated > >>>> ../support/timed_tests.sh "GLOG_v=1 MESOS_VERBOSE=1 make check > >>>> GTEST_SHUFFLE=1 GTEST_OUTPUT=xml:" 1800 > >>>> make[3]: *** [check-local] Terminated > >>>> make[2]: *** [check-am] Terminated > >>>> make[1]: *** [check] Terminated > >>>> Build step 'Execute shell' marked build as failure > >>>> Recording test results > >>>> > >>> > >>> > >> > > >
