[
https://issues.apache.org/jira/browse/MESOS-8796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761769#comment-16761769
]
Vinod Kone commented on MESOS-8796:
-----------------------------------
Saw this again on internal CI (on Mac).
{code}
[ RUN ] GroupTest.GroupPathWithRestrictivePerms
I0205 21:14:33.530055 296834496 zookeeper_test_server.cpp:156] Started
ZooKeeperTestServer on port 50946
2019-02-05 21:14:33,530:8369(0x7000036ae000):ZOO_INFO@log_env@753: Client
environment:zookeeper.version=zookeeper C client 3.4.8
2019-02-05 21:14:33,530:8369(0x7000036ae000):ZOO_INFO@log_env@757: Client
environment:host.name=Jenkinss-Mac-mini.local
2019-02-05 21:14:33,530:8369(0x7000036ae000):ZOO_INFO@log_env@764: Client
environment:os.name=Darwin
2019-02-05 21:14:33,530:8369(0x7000036ae000):ZOO_INFO@log_env@765: Client
environment:os.arch=18.2.0
2019-02-05 21:14:33,530:8369(0x7000036ae000):ZOO_INFO@log_env@766: Client
environment:os.version=Darwin Kernel Version 18.2.0: Mon Nov 12 20:24:46 PST
2018; root:xnu-4903.231.4~2/
RELEASE_X86_64
2019-02-05 21:14:33,530:8369(0x7000036ae000):ZOO_INFO@log_env@774: Client
environment:user.name=jenkins
2019-02-05 21:14:33,530:8369(0x7000036ae000):ZOO_INFO@log_env@782: Client
environment:user.home=/Users/jenkins
2019-02-05 21:14:33,530:8369(0x7000036ae000):ZOO_INFO@log_env@794: Client
environment:user.dir=/Users/jenkins/workspace/workspace/mesos/Mesos_CI-build/FLAG/SSL/label/mac/mesos/bui
ld
2019-02-05 21:14:33,530:8369(0x7000036ae000):ZOO_INFO@zookeeper_init@827:
Initiating client connection, host=127.0.0.1:50946 sessionTimeout=10000
watcher=0x1145565d0 sessionId=0 s
essionPasswd=<null> context=0x7fb3e0c9bc90 flags=0
2019-02-05 21:14:33,530:8369(0x700003fcf000):ZOO_INFO@check_events@1764:
initiated connection to server [127.0.0.1:50946]
2019-02-05 21:14:33,532:8369(0x700003fcf000):ZOO_INFO@check_events@1811:
session establishment complete on server [127.0.0.1:50946],
sessionId=0x168c13aa8b90000, negotiated timeou
t=10000
2019-02-05
21:14:36,875:8369(0x700003fcf000):ZOO_INFO@auth_completion_func@1327:
Authentication scheme digest succeeded
2019-02-05 21:14:36,878:8369(0x70000341f000):ZOO_INFO@log_env@753: Client
environment:zookeeper.version=zookeeper C client 3.4.8
2019-02-05 21:14:36,878:8369(0x70000341f000):ZOO_INFO@log_env@757: Client
environment:host.name=Jenkinss-Mac-mini.local
2019-02-05 21:14:36,878:8369(0x70000341f000):ZOO_INFO@log_env@764: Client
environment:os.name=Darwin
2019-02-05 21:14:36,878:8369(0x70000341f000):ZOO_INFO@log_env@765: Client
environment:os.arch=18.2.0
2019-02-05 21:14:36,878:8369(0x70000341f000):ZOO_INFO@log_env@766: Client
environment:os.version=Darwin Kernel Version 18.2.0: Mon Nov 12 20:24:46 PST
2018; root:xnu-4903.231.4~2/
RELEASE_X86_64
2019-02-05 21:14:36,878:8369(0x70000341f000):ZOO_INFO@log_env@774: Client
environment:user.name=jenkins
2019-02-05 21:14:36,878:8369(0x70000341f000):ZOO_INFO@log_env@782: Client
environment:user.home=/Users/jenkins
2019-02-05 21:14:36,878:8369(0x70000341f000):ZOO_INFO@log_env@794: Client
environment:user.dir=/Users/jenkins/workspace/workspace/mesos/Mesos_CI-build/FLAG/SSL/label/mac/mesos/bui
ld
2019-02-05 21:14:36,878:8369(0x70000341f000):ZOO_INFO@zookeeper_init@827:
Initiating client connection, host=127.0.0.1:50946 sessionTimeout=10000
watcher=0x1145565d0 sessionId=0 s
essionPasswd=<null> context=0x7fb3e0a4db10 flags=0
2019-02-05 21:14:36,879:8369(0x700004767000):ZOO_INFO@check_events@1764:
initiated connection to server [127.0.0.1:50946]
2019-02-05 21:14:36,880:8369(0x700004767000):ZOO_INFO@check_events@1811:
session establishment complete on server [127.0.0.1:50946],
sessionId=0x168c13aa8b90001, negotiated timeou
t=10000
I0205 21:14:36.880167 55189504 group.cpp:341] Group process
(zookeeper-group(48)@10.0.49.4:65013) connected to ZooKeeper
I0205 21:14:36.880213 55189504 group.cpp:831] Syncing group operations: queue
size (joins, cancels, datas) = (1, 0, 0)
I0205 21:14:36.880225 55189504 group.cpp:395] Authenticating with ZooKeeper
using digest
2019-02-05
21:14:40,222:8369(0x700004767000):ZOO_INFO@auth_completion_func@1327:
Authentication scheme digest succeeded
I0205 21:14:40.222224 55189504 group.cpp:419] Trying to create path
'/read-only' in ZooKeeper
2019-02-05 21:14:40,223:8369(0x7000036ae000):ZOO_INFO@log_env@753: Client
environment:zookeeper.version=zookeeper C client 3.4.8
2019-02-05 21:14:40,224:8369(0x7000036ae000):ZOO_INFO@log_env@757: Client
environment:host.name=Jenkinss-Mac-mini.local
2019-02-05 21:14:40,224:8369(0x7000036ae000):ZOO_INFO@log_env@764: Client
environment:os.name=Darwin
2019-02-05 21:14:40,224:8369(0x7000036ae000):ZOO_INFO@log_env@765: Client
environment:os.arch=18.2.0
2019-02-05 21:14:40,224:8369(0x7000036ae000):ZOO_INFO@log_env@766: Client
environment:os.version=Darwin Kernel Version 18.2.0: Mon Nov 12 20:24:46 PST
2018; root:xnu-4903.231.4~2/
RELEASE_X86_64
2019-02-05 21:14:40,224:8369(0x7000036ae000):ZOO_INFO@log_env@774: Client
environment:user.name=jenkins
2019-02-05 21:14:40,224:8369(0x7000036ae000):ZOO_INFO@log_env@782: Client
environment:user.home=/Users/jenkins
2019-02-05 21:14:40,224:8369(0x7000036ae000):ZOO_INFO@log_env@794: Client
environment:user.dir=/Users/jenkins/workspace/workspace/mesos/Mesos_CI-build/FLAG/SSL/label/mac/mesos/bui
ld
2019-02-05 21:14:40,224:8369(0x7000036ae000):ZOO_INFO@zookeeper_init@827:
Initiating client connection, host=127.0.0.1:50946 sessionTimeout=10000
watcher=0x1145565d0 sessionId=0 s
essionPasswd=<null> context=0x7fb3e03c00c0 flags=0
I0205 21:14:40.224323 55189504 group.cpp:758] Found non-sequence node
'writable' at '/read-only' in ZooKeeper
2019-02-05 21:14:40,224:8369(0x70000486d000):ZOO_INFO@check_events@1764:
initiated connection to server [127.0.0.1:50946]
2019-02-05 21:14:40,225:8369(0x70000486d000):ZOO_INFO@check_events@1811:
session establishment complete on server [127.0.0.1:50946],
sessionId=0x168c13aa8b90002, negotiated timeou
t=10000
I0205 21:14:40.225809 57335808 group.cpp:341] Group process
(zookeeper-group(49)@10.0.49.4:65013) connected to ZooKeeper
I0205 21:14:40.225847 57335808 group.cpp:831] Syncing group operations: queue
size (joins, cancels, datas) = (1, 0, 0)
I0205 21:14:40.225859 57335808 group.cpp:395] Authenticating with ZooKeeper
using digest
2019-02-05 21:14:49,320:8369(0x700004767000):ZOO_WARN@zookeeper_interest@1597:
Exceeded deadline by 5764ms
2019-02-05 21:14:49,320:8369(0x700003fcf000):ZOO_WARN@zookeeper_interest@1597:
Exceeded deadline by 5766ms
2019-02-05
21:14:49,320:8369(0x700004767000):ZOO_ERROR@handle_socket_error_msg@1702:
Socket [127.0.0.1:50946] zk retcode=-7, errno=60(Operation timed out):
connection to 127.0.0.1:50946 timed out (exceeded timeout by 2430ms)
2019-02-05
21:14:49,321:8369(0x700003fcf000):ZOO_ERROR@handle_socket_error_msg@1702:
Socket [127.0.0.1:50946] zk retcode=-7, errno=60(Operation timed out):
connection to 127.0.0.1:50946 timed out (exceeded timeout by 2432ms)
2019-02-05 21:14:49,321:8369(0x700004767000):ZOO_WARN@zookeeper_interest@1597:
Exceeded deadline by 5764ms
2019-02-05 21:14:49,321:8369(0x700003fcf000):ZOO_WARN@zookeeper_interest@1597:
Exceeded deadline by 5766ms
I0205 21:14:49.321324 54652928 group.cpp:452] Lost connection to ZooKeeper,
attempting to reconnect ...
2019-02-05 21:14:49,330:8369(0x70000486d000):ZOO_WARN@zookeeper_interest@1597:
Exceeded deadline by 5773ms
2019-02-05
21:14:49,330:8369(0x70000486d000):ZOO_ERROR@handle_socket_error_msg@1702:
Socket [127.0.0.1:50946] zk retcode=-7, errno=60(Operation timed out):
connection to 127.0.0.1:50946 timed out (exceeded timeout by 2438ms)
2019-02-05 21:14:49,330:8369(0x70000486d000):ZOO_WARN@zookeeper_interest@1597:
Exceeded deadline by 5773ms
I0205 21:14:49.330642 57335808 group.cpp:452] Lost connection to ZooKeeper,
attempting to reconnect ...
2019-02-05 21:14:52,664:8369(0x700003fcf000):ZOO_WARN@zookeeper_interest@1597:
Exceeded deadline by 9109ms
2019-02-05 21:14:52,664:8369(0x700004767000):ZOO_WARN@zookeeper_interest@1597:
Exceeded deadline by 9107ms
2019-02-05 21:14:52,664:8369(0x70000486d000):ZOO_WARN@zookeeper_interest@1597:
Exceeded deadline by 9106ms
2019-02-05 21:14:52,664:8369(0x700004767000):ZOO_INFO@check_events@1764:
initiated connection to server [127.0.0.1:50946]
2019-02-05 21:14:52,664:8369(0x700003fcf000):ZOO_INFO@check_events@1764:
initiated connection to server [127.0.0.1:50946]
2019-02-05 21:14:52,664:8369(0x70000486d000):ZOO_INFO@check_events@1764:
initiated connection to server [127.0.0.1:50946]
2019-02-05
21:14:52,665:8369(0x700003fcf000):ZOO_ERROR@handle_socket_error_msg@1800:
Socket [127.0.0.1:50946] zk retcode=-112, errno=70(Stale NFS file handle):
sessionId=0x168c13aa8b90000 has expired.
2019-02-05
21:14:52,665:8369(0x700004767000):ZOO_ERROR@handle_socket_error_msg@1800:
Socket [127.0.0.1:50946] zk retcode=-112, errno=70(Stale NFS file handle):
sessionId=0x168c13aa8b90001 has expired.
I0205 21:14:52.665789 56799232 group.cpp:511] ZooKeeper session expired
2019-02-05 21:14:52,665:8369(0x70000362b000):ZOO_INFO@zookeeper_close@2579:
Freeing zookeeper resources for sessionId=0x168c13aa8b90001
2019-02-05
21:14:52,665:8369(0x70000486d000):ZOO_ERROR@handle_socket_error_msg@1800:
Socket [127.0.0.1:50946] zk retcode=-112, errno=70(Stale NFS file handle):
sessionId=0x168c13aa8b90002 has expired.
2019-02-05 21:14:52,666:8369(0x700003525000):ZOO_INFO@log_env@753: Client
environment:zookeeper.version=zookeeper C client 3.4.8
I0205 21:14:52.666157 55189504 group.cpp:511] ZooKeeper session expired
2019-02-05 21:14:52,666:8369(0x700003525000):ZOO_INFO@log_env@757: Client
environment:host.name=Jenkinss-Mac-mini.local
2019-02-05 21:14:52,666:8369(0x700003525000):ZOO_INFO@log_env@764: Client
environment:os.name=Darwin
2019-02-05 21:14:52,666:8369(0x700003525000):ZOO_INFO@log_env@765: Client
environment:os.arch=18.2.0
2019-02-05 21:14:52,666:8369(0x700003525000):ZOO_INFO@log_env@766: Client
environment:os.version=Darwin Kernel Version 18.2.0: Mon Nov 12 20:24:46 PST
2018; root:xnu-4903.231.4~2/RELEASE_X86_64
2019-02-05 21:14:52,666:8369(0x700003525000):ZOO_INFO@log_env@774: Client
environment:user.name=jenkins
2019-02-05 21:14:52,666:8369(0x700003525000):ZOO_INFO@log_env@782: Client
environment:user.home=/Users/jenkins
2019-02-05 21:14:52,666:8369(0x7000034a2000):ZOO_INFO@zookeeper_close@2579:
Freeing zookeeper resources for sessionId=0x168c13aa8b90002
2019-02-05 21:14:52,666:8369(0x700003525000):ZOO_INFO@log_env@794: Client
environment:user.dir=/Users/jenkins/workspace/workspace/mesos/Mesos_CI-build/FLAG/SSL/label/mac/mesos/build
2019-02-05 21:14:52,666:8369(0x700003525000):ZOO_INFO@zookeeper_init@827:
Initiating client connection, host=127.0.0.1:50946 sessionTimeout=10000
watcher=0x1145565d0 sessionId=0 sessionPasswd=<null> context=0x7fb3e0cad090
flags=0
2019-02-05 21:14:52,666:8369(0x7000035a8000):ZOO_INFO@log_env@753: Client
environment:zookeeper.version=zookeeper C client 3.4.8
2019-02-05 21:14:52,666:8369(0x7000035a8000):ZOO_INFO@log_env@757: Client
environment:host.name=Jenkinss-Mac-mini.local
2019-02-05 21:14:52,666:8369(0x7000035a8000):ZOO_INFO@log_env@764: Client
environment:os.name=Darwin
2019-02-05 21:14:52,666:8369(0x7000035a8000):ZOO_INFO@log_env@765: Client
environment:os.arch=18.2.0
2019-02-05 21:14:52,666:8369(0x7000035a8000):ZOO_INFO@log_env@766: Client
environment:os.version=Darwin Kernel Version 18.2.0: Mon Nov 12 20:24:46 PST
2018; root:xnu-4903.231.4~2/:
RELEASE_X86_64
2019-02-05 21:14:52,666:8369(0x7000035a8000):ZOO_INFO@log_env@774: Client
environment:user.name=jenkins
2019-02-05 21:14:52,666:8369(0x7000035a8000):ZOO_INFO@log_env@782: Client
environment:user.home=/Users/jenkins
2019-02-05 21:14:52,666:8369(0x7000035a8000):ZOO_INFO@log_env@794: Client
environment:user.dir=/Users/jenkins/workspace/workspace/mesos/Mesos_CI-build/FLAG/SSL/label/mac/mesos/build
2019-02-05 21:14:52,666:8369(0x7000035a8000):ZOO_INFO@zookeeper_init@827:
Initiating client connection, host=127.0.0.1:50946 sessionTimeout=10000
watcher=0x1145565d0 sessionId=0 sessionPasswd=<null> context=0x7fb3e03c00c0
flags=0
2019-02-05 21:14:52,666:8369(0x700004767000):ZOO_INFO@check_events@1764:
initiated connection to server [127.0.0.1:50946]
2019-02-05 21:14:52,666:8369(0x70000486d000):ZOO_INFO@check_events@1764:
initiated connection to server [127.0.0.1:50946]
2019-02-05 21:14:52,667:8369(0x700004767000):ZOO_INFO@check_events@1811:
session establishment complete on server [127.0.0.1:50946],
sessionId=0x168c13aa8b90003, negotiated timeout=10000
I0205 21:14:52.668120 54116352 group.cpp:341] Group process
(zookeeper-group(48)@10.0.49.4:65013) connected to ZooKeeper
I0205 21:14:52.668161 54116352 group.cpp:831] Syncing group operations: queue
size (joins, cancels, datas) = (0, 0, 0)
I0205 21:14:52.668203 54116352 group.cpp:395] Authenticating with ZooKeeper
using digest
2019-02-05 21:14:52,668:8369(0x70000486d000):ZOO_INFO@check_events@1811:
session establishment complete on server [127.0.0.1:50946],
sessionId=0x168c13aa8b90004, negotiated timeout=10000
I0205 21:14:52.668565 54652928 group.cpp:341] Group process
(zookeeper-group(49)@10.0.49.4:65013) connected to ZooKeeper
I0205 21:14:52.668601 54652928 group.cpp:831] Syncing group operations: queue
size (joins, cancels, datas) = (1, 0, 0)
I0205 21:14:52.668615 54652928 group.cpp:395] Authenticating with ZooKeeper
using digest
../../src/tests/group_tests.cpp:333: Failure
Failed to wait 15secs for failedGroup2.join("fail")
2019-02-05 21:14:56,010:8369(0x700004767000):ZOO_WARN@zookeeper_interest@1597:
Exceeded deadline by 11ms
2019-02-05
21:14:56,010:8369(0x70000486d000):ZOO_INFO@auth_completion_func@1327:
Authentication scheme digest succeeded
2019-02-05
21:14:56,010:8369(0x700004767000):ZOO_INFO@auth_completion_func@1327:
Authentication scheme digest succeeded
I0205 21:14:56.011027 54652928 group.cpp:419] Trying to create path
'/read-only/new' in ZooKeeper
I0205 21:14:56.011297 54116352 group.cpp:419] Trying to create path
'/read-only' in ZooKeeper
I0205 21:14:56.012171 54116352 group.cpp:758] Found non-sequence node
'writable' at '/read-only' in ZooKeeper
E0205 21:14:56.012761 54652928 group.cpp:961] Group aborting: Failed to create
'/read-only/new' in ZooKeeper: not authenticated
2019-02-05 21:14:56,012:8369(0x70000341f000):ZOO_INFO@zookeeper_close@2562:
Closing zookeeper sessionId=0x168c13aa8b90004 to [127.0.0.1:50946]
2019-02-05 21:14:56,013:8369(0x111b155c0):ZOO_INFO@zookeeper_close@2562:
Closing zookeeper sessionId=0x168c13aa8b90003 to [127.0.0.1:50946]
2019-02-05 21:14:56,013:8369(0x111b155c0):ZOO_INFO@zookeeper_close@2579:
Freeing zookeeper resources for sessionId=0x168c13aa8b90000
I0205 21:14:56.013558 296834496 zookeeper_test_server.cpp:116] Shutting down
ZooKeeperTestServer on port 50946
[ FAILED ] GroupTest.GroupPathWithRestrictivePerms (22486 ms)
{code}
> Some GroupTest.* are flaky on Mac.
> ----------------------------------
>
> Key: MESOS-8796
> URL: https://issues.apache.org/jira/browse/MESOS-8796
> Project: Mesos
> Issue Type: Bug
> Components: test
> Affects Versions: 1.6.0, 1.8.0
> Environment: Mac OS with SSL enabled
> Reporter: Alexander Rukletsov
> Priority: Minor
> Labels: flaky, flaky-test, zookeeper
> Attachments: GroupJoinWithDisconnect-badrun.txt,
> GroupPathWithRestrictivePerms-badrun.txt,
> GroupPathWithRestrictivePerms-goodrun.txt,
> GroupTest.GroupPathWithRest-another-bad-run.txt, RetryableErrors-badrun.txt
>
>
> I see some failures related to zookeeper on our Mac machine. Current list of
> failing tests:
> {noformat}
> GroupTest.GroupPathWithRestrictivePerms
> GroupTest.RetryableErrors
> {noformat}
> Full logs attached.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
