[
https://issues.apache.org/jira/browse/MESOS-455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13654727#comment-13654727
]
Vinod Kone commented on MESOS-455:
----------------------------------
another instance of this. didn't have verbose logging turned on unfortunately.
{code}
[ RUN ] ZooKeeperTest.MasterDetectorShutdownNetwork
2013-05-10
18:45:15,778:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1603: Socket
[127.0.0.1:46157] zk retcode=-4, errno=112(Host is down): failed while
receiving a server response
2013-05-10
18:45:19,116:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1579: Socket
[127.0.0.1:46157] zk retcode=-4, errno=111(Connection refused): server refused
to accept the client
2013-05-10
18:45:22,453:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1579: Socket
[127.0.0.1:46157] zk retcode=-4, errno=111(Connection refused): server refused
to accept the client
2013-05-10
18:45:25,790:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1579: Socket
[127.0.0.1:46157] zk retcode=-4, errno=111(Connection refused): server refused
to accept the client
2013-05-10
18:45:29,126:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1579: Socket
[127.0.0.1:46157] zk retcode=-4, errno=111(Connection refused): server refused
to accept the client
2013-05-10
18:45:32,463:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1579: Socket
[127.0.0.1:46157] zk retcode=-4, errno=111(Connection refused): server refused
to accept the client
2013-05-10
18:45:35,800:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1579: Socket
[127.0.0.1:46157] zk retcode=-4, errno=111(Connection refused): server refused
to accept the client
2013-05-10
18:45:39,136:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1579: Socket
[127.0.0.1:46157] zk retcode=-4, errno=111(Connection refused): server refused
to accept the client
2013-05-10
18:45:42,473:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1579: Socket
[127.0.0.1:46157] zk retcode=-4, errno=111(Connection refused): server refused
to accept the client
2013-05-10
18:45:45,809:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1579: Socket
[127.0.0.1:46157] zk retcode=-4, errno=111(Connection refused): server refused
to accept the client
2013-05-10
18:45:49,146:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1579: Socket
[127.0.0.1:46157] zk retcode=-4, errno=111(Connection refused): server refused
to accept the client
2013-05-10
18:45:52,483:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1579: Socket
[127.0.0.1:46157] zk retcode=-4, errno=111(Connection refused): server refused
to accept the client
2013-05-10
18:45:55,819:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1579: Socket
[127.0.0.1:46157] zk retcode=-4, errno=111(Connection refused): server refused
to accept the client
2013-05-10
18:45:59,156:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1579: Socket
[127.0.0.1:46157] zk retcode=-4, errno=111(Connection refused): server refused
to accept the client
2013-05-10
18:46:02,493:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1579: Socket
[127.0.0.1:46157] zk retcode=-4, errno=111(Connection refused): server refused
to accept the client
2013-05-10
18:46:05,829:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1579: Socket
[127.0.0.1:46157] zk retcode=-4, errno=111(Connection refused): server refused
to accept the client
2013-05-10
18:46:09,166:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1579: Socket
[127.0.0.1:46157] zk retcode=-4, errno=111(Connection refused): server refused
to accept the client
2013-05-10
18:46:12,502:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1579: Socket
[127.0.0.1:46157] zk retcode=-4, errno=111(Connection refused): server refused
to accept the client
2013-05-10
18:46:15,839:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1579: Socket
[127.0.0.1:46157] zk retcode=-4, errno=111(Connection refused): server refused
to accept the client
2013-05-10
18:46:19,175:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1579: Socket
[127.0.0.1:46157] zk retcode=-4, errno=111(Connection refused): server refused
to accept the client
2013-05-10
18:46:22,512:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1579: Socket
[127.0.0.1:46157] zk retcode=-4, errno=111(Connection refused): server refused
to accept the client
2013-05-10
18:46:25,849:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1579: Socket
[127.0.0.1:46157] zk retcode=-4, errno=111(Connection refused): server refused
to accept the client
2013-05-10
18:46:29,185:36011(0x5001e940):ZOO_ERROR@handle_socket_error_msg@1579: Socket
[127.0.0.1:46157] zk retcode=-4, errno=111(Connection refused): server refused
to accept the client
{code}
gdb stack trace
{code}
#0 0x00007fb620be4ee9 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00007fb621435b0b in arrive (this=0x1a9f100, pid=<value optimized out>) at
../../../third_party/libprocess/src/gate.hpp:73
#2 process::ProcessManager::wait (this=0x1a9f100, pid=<value optimized out>)
at ../../../third_party/libprocess/src/process.cpp:2664
#3 0x00007fb6214415b5 in process::wait (pid=..., duration=...) at
../../../third_party/libprocess/src/process.cpp:3104
#4 0x00007fb621441539 in wait (pid=..., duration=...) at
../../../third_party/libprocess/include/process/process.hpp:351
#5 process::wait (pid=..., duration=...) at
../../../third_party/libprocess/src/process.cpp:3111
#6 0x00007fb6214288fa in process::Latch::await (this=0x7fb6100b7140,
duration=...) at ../../../third_party/libprocess/src/latch.cpp:45
#7 0x00000000004be69a in process::Future<Nothing>::await (this=0x7fff3b1e8de0,
duration=...) at ../../third_party/libprocess/include/process/future.hpp:611
#8 0x00000000004f05b6 in AwaitAssertReady<Nothing> (expr=0x91d3a6
"noMasterDetected", actual=..., duration=...) at
../../third_party/libprocess/include/process/gtest.hpp:49
#9 0x00000000007fd7be in
ZooKeeperTest_MasterDetectorShutdownNetwork_Test::TestBody
(this=0x7fb618012200) at ../../src/tests/zookeeper_tests.cpp:288
#10 0x00000000008488bb in
testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>
(object=0x7fb618012200, method=&virtual testing::Test::TestBody(),
location=0x92ec77 "the test body")
at gmock-1.6.0/gtest/src/gtest.cc:2126
#11 0x000000000083c749 in testing::Test::Run (this=0x7fb618012200) at
gmock-1.6.0/gtest/src/gtest.cc:2162
#12 0x000000000083c885 in testing::TestInfo::Run (this=0x1a73340) at
gmock-1.6.0/gtest/src/gtest.cc:2338
#13 0x000000000083c93d in testing::TestCase::Run (this=0x1a72f90) at
gmock-1.6.0/gtest/src/gtest.cc:2445
#14 0x000000000083fb1d in testing::internal::UnitTestImpl::RunAllTests
(this=0x1a6e950) at gmock-1.6.0/gtest/src/gtest.cc:4237
#15 0x0000000000840515 in
testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,
bool> (object=<value optimized out>, method=
(bool (testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl
*)) 0x83f8a0 <testing::internal::UnitTestImpl::RunAllTests()>,
location=0xffffffffffffffff <Address 0xffffffffffffffff out of bounds>)
at gmock-1.6.0/gtest/src/gtest.cc:2090
#16 0x000000000084843e in
testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,
bool> (object=0x1a6e950, method=
(bool (testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl
*)) 0x83f8a0 <testing::internal::UnitTestImpl::RunAllTests()>,
location=0x930900 "auxiliary test code (environments or event listeners)")
at gmock-1.6.0/gtest/src/gtest.cc:2126
#17 0x000000000083bcca in testing::UnitTest::Run (this=<value optimized out>)
at gmock-1.6.0/gtest/src/gtest.cc:3874
#18 0x000000000047bea9 in main (argc=1, argv=<value optimized out>) at
../../src/tests/main.cpp:103
{code}
> ZooKeeperTest.MasterDetectorShutdownNetwork runs forever
> --------------------------------------------------------
>
> Key: MESOS-455
> URL: https://issues.apache.org/jira/browse/MESOS-455
> Project: Mesos
> Issue Type: Bug
> Reporter: Thomas Marshall
>
> https://hadrian.millennium.berkeley.edu/jenkins/job/Mesos/116/consoleFull
> [ RUN ] ZooKeeperTest.MasterDetectorShutdownNetwork
> I0426 11:59:53.030719 8083 jvm.cpp:34] looking up method
> <init>(Ljava/lang/String;)V
> I0426 11:59:53.030918 8083 jvm.cpp:34] looking up method <init>(I)V
> I0426 11:59:53.031435 8083 jvm.cpp:34] looking up method
> <init>(Ljava/net/InetSocketAddress;)V
> I0426 11:59:53.031880 8083 jvm.cpp:34] looking up method
> startup(Lorg/apache/zookeeper/server/ZooKeeperServer;)V
> I0426 11:59:53.032313 8083 jvm.cpp:34] looking up method isAlive()Z
> I0426 11:59:53.032734 8083 jvm.cpp:34] looking up method shutdown()V
> I0426 11:59:53.035411 8083 jvm.cpp:34] looking up method
> <init>(Ljava/io/File;Ljava/io/File;)V
> I0426 11:59:53.036499 8083 jvm.cpp:34] looking up method <init>()V
> I0426 11:59:53.036887 8083 jvm.cpp:34] looking up method
> <init>(Lorg/apache/zookeeper/server/persistence/FileTxnSnapLog;Lorg/apache/zookeeper/server/ZooKeeperServer$DataTreeBuilder;)V
> I0426 11:59:53.037300 8083 jvm.cpp:34] looking up method getClientPort()I
> I0426 11:59:53.037652 8083 jvm.cpp:34] looking up method closeSession(J)V
> I0426 11:59:53.041681 8083 zookeeper_test_server.cpp:212] Started
> ZooKeeperTestServer on port 40364
> I0426 11:59:53.049566 8103 detector.cpp:283] Master detector
> ((41)@10.123.1.50:50879) connected to ZooKeeper ...
> I0426 11:59:53.049686 8103 detector.cpp:300] Trying to create path '/mesos'
> in ZooKeeper
> I0426 11:59:53.061887 8103 detector.cpp:330] Created ephemeral/sequence
> znode at '/mesos/0000000000'
> I0426 11:59:53.063654 8103 detector.cpp:485] Master detector
> ((41)@10.123.1.50:50879) found 1 registered masters
> I0426 11:59:53.070186 8103 detector.cpp:531] Master detector
> ((41)@10.123.1.50:50879) got new master pid: (41)@10.123.1.50:50879
> 2013-04-26 11:59:53,073:8083(0x7fabacb43700I0426 11:59:53.074367 8083
> zookeeper_test_server.cpp:195] Shutdown ZooKeeperTestServer on port 40364
> ):ZOO_ERROR@handle_socket_error_msg@1603: Socket [127.0.0.1:40364] zk
> retcode=-4, errno=112(Host is down): failed while receiving a server response
> I0426 11:59:53.076418 8105 detector.cpp:374] Master detector
> ((41)@10.123.1.50:50879) lost connection to ZooKeeper, attempting to
> reconnect ...
> 2013-04-26
> 11:59:56,412:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 11:59:59,750:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:00:03,088:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:00:06,425:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:00:09,762:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:00:13,099:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:00:16,437:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:00:19,775:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:00:23,113:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:00:26,450:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:00:29,788:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:00:33,126:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:00:36,463:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:00:39,800:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:00:43,137:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:00:46,474:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:00:49,811:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:00:53,148:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:00:56,485:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:00:59,822:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:01:03,160:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:01:06,496:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:01:09,833:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:01:13,170:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:01:16,507:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:01:19,844:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:01:23,181:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:01:26,518:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:01:29,854:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:01:33,191:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:01:36,528:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:01:39,865:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:01:43,202:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:01:46,538:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:01:49,875:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:01:53,212:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2013-04-26
> 12:01:56,549:8083(0x7fabacb43700):ZOO_ERROR@handle_socket_error_msg@1579:
> Socket [127.0.0.1:40364] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
