[
https://issues.apache.org/jira/browse/MESOS-6784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15746259#comment-15746259
]
Jie Yu commented on MESOS-6784:
-------------------------------
Another data point with different log:
{noformat}
[20:16:40] : [Step 11/11] [ RUN ]
IOSwitchboardTest.KillSwitchboardContainerDestroyed
[20:16:40] : [Step 11/11] I1213 20:16:40.465116 26604
containerizer.cpp:220] Using isolation: posix/cpu,filesystem/posix,network/cni
[20:16:40] : [Step 11/11] I1213 20:16:40.465904 26624
containerizer.cpp:594] Recovering containerizer
[20:16:40] : [Step 11/11] I1213 20:16:40.466194 26623 provisioner.cpp:253]
Provisioner recovery complete
[20:16:40] : [Step 11/11] I1213 20:16:40.466544 26625
containerizer.cpp:986] Starting container 8fc415c0-74cc-4c13-8bcf-1f3dfc7a993b
for executor 'executor' of framework
[20:16:40] : [Step 11/11] I1213 20:16:40.467177 26622 switchboard.cpp:430]
Allocated pseudo terminal '/dev/pts/0' for container
8fc415c0-74cc-4c13-8bcf-1f3dfc7a993b
[20:16:40] : [Step 11/11] I1213 20:16:40.467337 26622 switchboard.cpp:567]
Launching 'mesos-io-switchboard' with flags '--heartbeat_interval="30secs"
--help="false"
--socket_address="/tmp/mesos-io-switchboard-2b923443-e3d2-4513-9442-31536611ea28"
--stderr_from_fd="13" --stderr_to_fd="2" --stdin_to_fd="13"
--stdout_from_fd="13" --stdout_to_fd="1" --tty="true"
--wait_for_connection="false"' for container
8fc415c0-74cc-4c13-8bcf-1f3dfc7a993b
[20:16:40] : [Step 11/11] I1213 20:16:40.469470 26622 switchboard.cpp:597]
Created I/O switchboard server (pid: 11965) listening on socket file
'/tmp/mesos-io-switchboard-2b923443-e3d2-4513-9442-31536611ea28' for container
8fc415c0-74cc-4c13-8bcf-1f3dfc7a993b
[20:16:40] : [Step 11/11] I1213 20:16:40.470201 26619
containerizer.cpp:1535] Launching 'mesos-containerizer' with flags
'--help="false" --launch_info="{"command":{"shell":true,"value":"sleep
1000"},"environment":{"variables":[{"name":"MESOS_SANDBOX","value":"\/mnt\/teamcity\/temp\/buildTmp\/IOSwitchboardTest_KillSwitchboardContainerDestroyed_gYP6xR"}]},"err":{"fd":14,"type":"FD"},"in":{"fd":14,"type":"FD"},"out":{"fd":14,"type":"FD"},"tty_slave_path":"\/dev\/pts\/0","working_directory":"\/mnt\/teamcity\/temp\/buildTmp\/IOSwitchboardTest_KillSwitchboardContainerDestroyed_gYP6xR"}"
--pipe_read="13" --pipe_write="15"
--runtime_directory="/mnt/teamcity/temp/buildTmp/IOSwitchboardTest_KillSwitchboardContainerDestroyed_3d8l0O/containers/8fc415c0-74cc-4c13-8bcf-1f3dfc7a993b"
--unshare_namespace_mnt="false"'
[20:16:40] : [Step 11/11] I1213 20:16:40.471458 26619 launcher.cpp:133]
Forked child with pid '11966' for container
'8fc415c0-74cc-4c13-8bcf-1f3dfc7a993b'
[20:16:40] : [Step 11/11] I1213 20:16:40.471776 26619
containerizer.cpp:1634] Checkpointing container's forked pid 11966 to
'/mnt/teamcity/temp/buildTmp/IOSwitchboardTest_KillSwitchboardContainerDestroyed_fdQTsM/meta/slaves/frameworks/executors/executor/runs/8fc415c0-74cc-4c13-8bcf-1f3dfc7a993b/pids/forked.pid'
[20:16:40] : [Step 11/11] I1213 20:16:40.472676 26621 fetcher.cpp:349]
Starting to fetch URIs for container: 8fc415c0-74cc-4c13-8bcf-1f3dfc7a993b,
directory:
/mnt/teamcity/temp/buildTmp/IOSwitchboardTest_KillSwitchboardContainerDestroyed_gYP6xR
[20:16:40] : [Step 11/11] E1213 20:16:40.547350 26625 switchboard.cpp:880]
Unexpected termination of I/O switchboard server: 'IOSwitchboard' exited with
signal: Killed for container 8fc415c0-74cc-4c13-8bcf-1f3dfc7a993b
[20:16:40] : [Step 11/11] I1213 20:16:40.547364 26620
containerizer.cpp:2493] Container 8fc415c0-74cc-4c13-8bcf-1f3dfc7a993b has
reached its limit for resource {} and will be terminated
[20:16:40] : [Step 11/11] I1213 20:16:40.547385 26620
containerizer.cpp:2113] Destroying container
8fc415c0-74cc-4c13-8bcf-1f3dfc7a993b in RUNNING state
[20:16:40] : [Step 11/11] I1213 20:16:40.547490 26620 launcher.cpp:149]
Asked to destroy container 8fc415c0-74cc-4c13-8bcf-1f3dfc7a993b
[20:16:40] : [Step 11/11] I1213 20:16:40.552752 26620
containerizer.cpp:2476] Container 8fc415c0-74cc-4c13-8bcf-1f3dfc7a993b has
exited
[20:16:40] : [Step 11/11] E1213 20:16:40.553004 26624 switchboard.cpp:801]
Failed to remove unix domain socket file
'/tmp/mesos-io-switchboard-2b923443-e3d2-4513-9442-31536611ea28' for container
'8fc415c0-74cc-4c13-8bcf-1f3dfc7a993b': No such file or directory
[20:16:40] : [Step 11/11] I1213 20:16:40.553323 26624 provisioner.cpp:324]
Ignoring destroy request for unknown container
8fc415c0-74cc-4c13-8bcf-1f3dfc7a993b
[20:16:40] : [Step 11/11]
../../src/tests/containerizer/io_switchboard_tests.cpp:668: Failure
[20:16:40] : [Step 11/11] Expecting WIFSIGNALED(wait.get()->status()) but
WIFEXITED(wait.get()->status()) is true and WEXITSTATUS(wait.get()->status())
is 1
[20:16:40] : [Step 11/11] [ FAILED ]
IOSwitchboardTest.KillSwitchboardContainerDestroyed (100 ms)
{noformat}
> IOSwitchboardTest.KillSwitchboardContainerDestroyed is flaky
> ------------------------------------------------------------
>
> Key: MESOS-6784
> URL: https://issues.apache.org/jira/browse/MESOS-6784
> Project: Mesos
> Issue Type: Bug
> Components: agent
> Reporter: Neil Conway
> Assignee: Kevin Klues
> Labels: mesosphere
>
> {noformat}
> [ RUN ] IOSwitchboardTest.KillSwitchboardContainerDestroyed
> I1212 13:57:02.641043 2211 containerizer.cpp:220] Using isolation:
> posix/cpu,filesystem/posix,network/cni
> W1212 13:57:02.641438 2211 backend.cpp:76] Failed to create 'overlay'
> backend: OverlayBackend requires root privileges, but is running as user nrc
> W1212 13:57:02.641559 2211 backend.cpp:76] Failed to create 'bind' backend:
> BindBackend requires root privileges
> I1212 13:57:02.642822 2268 containerizer.cpp:594] Recovering containerizer
> I1212 13:57:02.643975 2253 provisioner.cpp:253] Provisioner recovery complete
> I1212 13:57:02.644953 2255 containerizer.cpp:986] Starting container
> 09e87380-00ab-4987-83c9-fa1c5d86717f for executor 'executor' of framework
> I1212 13:57:02.647004 2245 switchboard.cpp:430] Allocated pseudo terminal
> '/dev/pts/54' for container 09e87380-00ab-4987-83c9-fa1c5d86717f
> I1212 13:57:02.652305 2245 switchboard.cpp:596] Created I/O switchboard
> server (pid: 2705) listening on socket file
> '/tmp/mesos-io-switchboard-b4af1c92-6633-44f3-9d35-e0e36edaf70a' for
> container 09e87380-00ab-4987-83c9-fa1c5d86717f
> I1212 13:57:02.655513 2267 launcher.cpp:133] Forked child with pid '2706'
> for container '09e87380-00ab-4987-83c9-fa1c5d86717f'
> I1212 13:57:02.655732 2267 containerizer.cpp:1621] Checkpointing container's
> forked pid 2706 to
> '/tmp/IOSwitchboardTest_KillSwitchboardContainerDestroyed_Me5CRx/meta/slaves/frameworks/executors/executor/runs/09e87380-00ab-4987-83c9-fa1c5d86717f/pids/forked.pid'
> I1212 13:57:02.726306 2265 containerizer.cpp:2463] Container
> 09e87380-00ab-4987-83c9-fa1c5d86717f has exited
> I1212 13:57:02.726352 2265 containerizer.cpp:2100] Destroying container
> 09e87380-00ab-4987-83c9-fa1c5d86717f in RUNNING state
> E1212 13:57:02.726495 2243 switchboard.cpp:861] Unexpected termination of
> I/O switchboard server: 'IOSwitchboard' exited with signal: Killed for
> container 09e87380-00ab-4987-83c9-fa1c5d86717f
> I1212 13:57:02.726563 2265 launcher.cpp:149] Asked to destroy container
> 09e87380-00ab-4987-83c9-fa1c5d86717f
> E1212 13:57:02.783607 2228 switchboard.cpp:799] Failed to remove unix domain
> socket file '/tmp/mesos-io-switchboard-b4af1c92-6633-44f3-9d35-e0e36edaf70a'
> for container '09e87380-00ab-4987-83c9-fa1c5d86717f': No such file or
> directory
> ../../mesos/src/tests/containerizer/io_switchboard_tests.cpp:661: Failure
> Value of: wait.get()->reasons().size() == 1
> Actual: false
> Expected: true
> *** Aborted at 1481579822 (unix time) try "date -d @1481579822" if you are
> using GNU date ***
> PC: @ 0x1bf16d0 testing::UnitTest::AddTestPartResult()
> *** SIGSEGV (@0x0) received by PID 2211 (TID 0x7faed7d078c0) from PID 0;
> stack trace: ***
> @ 0x7faecf855100 (unknown)
> @ 0x1bf16d0 testing::UnitTest::AddTestPartResult()
> @ 0x1be6247 testing::internal::AssertHelper::operator=()
> @ 0x19ed751
> mesos::internal::tests::IOSwitchboardTest_KillSwitchboardContainerDestroyed_Test::TestBody()
> @ 0x1c0ed8c
> testing::internal::HandleSehExceptionsInMethodIfSupported<>()
> @ 0x1c09e74
> testing::internal::HandleExceptionsInMethodIfSupported<>()
> @ 0x1beb505 testing::Test::Run()
> @ 0x1bebc88 testing::TestInfo::Run()
> @ 0x1bec2ce testing::TestCase::Run()
> @ 0x1bf2ba8 testing::internal::UnitTestImpl::RunAllTests()
> @ 0x1c0f9b1
> testing::internal::HandleSehExceptionsInMethodIfSupported<>()
> @ 0x1c0a9f2
> testing::internal::HandleExceptionsInMethodIfSupported<>()
> @ 0x1bf18ee testing::UnitTest::Run()
> @ 0x11bc9e3 RUN_ALL_TESTS()
> @ 0x11bc599 main
> @ 0x7faece663b15 __libc_start_main
> @ 0xa9c219 (unknown)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
