[
https://issues.apache.org/jira/browse/ZOOKEEPER-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthew Hertz updated ZOOKEEPER-3490:
-------------------------------------
Description:
Hi,
We have a 3 node Zookeeper cluster. There are a number of znode's on the leader
that are not visible on the followers.
{code:java}
$ zkCli -server <server 1> (follower)
[zk: <server 1>(CONNECTED) 0] get
/pyMkdProducer/SNAP/lock/c4a62c9fdfdc412fac3818bbb2af3a0f__lock__0000000040
abcd.company.com:<built-in function getpid>
cZxid = 0xf00061d68
ctime = Thu Nov 01 12:40:33 GMT 2018
mZxid = 0xf00061d68
mtime = Thu Nov 01 12:40:33 GMT 2018
pZxid = 0xf00061d68
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x500be5318d60407
dataLength = 58
numChildren = 0
{code}
{code:java}
$ zkCli -server <server 2> (leader)
[zk: <server2>(CONNECTED) 0] get
/pyMkdProducer/SNAP/lock/c4a62c9fdfdc412fac3818bbb2af3a0f__lock__0000000040
Node does not exist:
/pyMkdProducer/SNAP/lock/c4a62c9fdfdc412fac3818bbb2af3a0f__lock__0000000040
{code}
{code:java}
$ zkCli -server <server 3> (follower)
[zk: <server3>(CONNECTED) 0] get
/pyMkdProducer/SNAP/lock/c4a62c9fdfdc412fac3818bbb2af3a0f__lock__0000000040
abcd.company.com:<built-in function getpid>
cZxid = 0xf00061d68
ctime = Thu Nov 01 12:40:33 GMT 2018
mZxid = 0xf00061d68
mtime = Thu Nov 01 12:40:33 GMT 2018
pZxid = 0xf00061d68
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x500be5318d60407
dataLength = 58
numChildren = 0
{code}
These nodes are ephemeral nodes. The sessions no longer exist. There are 6
znodes in this 'inconsistent' state. The cluster is currently connected - there
are no networking partitions currently.
We're at a loss for how to both debug and fix this. Restarting the Zookeeper
followers presumably will not help? Are all nodes ever force-synced from the
leader?
Help would be appreciated. If any more information would be helpful it can be
provided, however we will likely have to resolve this issue one way or another
in the near future.
Thanks
was:
Hi,
We have a 3 node Zookeeper cluster. There are a number of znode's on the leader
that are not visible on the followers.
{code:java}
$ zkCli -server <server 1> (follower)
[zk: <server 1>(CONNECTED) 0] get
/pyMkdProducer/SNAP/lock/c4a62c9fdfdc412fac3818bbb2af3a0f__lock__0000000040
abcd.company.com:<built-in function getpid>
cZxid = 0xf00061d68
ctime = Thu Nov 01 12:40:33 GMT 2018
mZxid = 0xf00061d68
mtime = Thu Nov 01 12:40:33 GMT 2018
pZxid = 0xf00061d68
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x500be5318d60407
dataLength = 58
numChildren = 0
{code}
{code:java}
$ zkCli -server <server 2> (leader)
[zk: <server2>(CONNECTED) 0] get
/pyMkdProducer/SNAP/lock/c4a62c9fdfdc412fac3818bbb2af3a0f__lock__0000000040
Node does not exist:
/pyMkdProducer/SNAP/lock/c4a62c9fdfdc412fac3818bbb2af3a0f__lock__0000000040
{code}
{code:java}
$ zkCli -server <server 3> (follower)
[zk: <server3>(CONNECTED) 0] get
/pyMkdProducer/SNAP/lock/c4a62c9fdfdc412fac3818bbb2af3a0f__lock__0000000040
abcd.company.com:<built-in function getpid>
cZxid = 0xf00061d68
ctime = Thu Nov 01 12:40:33 GMT 2018
mZxid = 0xf00061d68
mtime = Thu Nov 01 12:40:33 GMT 2018
pZxid = 0xf00061d68
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x500be5318d60407
dataLength = 58
numChildren = 0
{code}
These nodes are ephemeral nodes. The sessions no longer exist. There are 6
znodes in this 'inconsistent' state. The cluster is currently connected - there
are no networking partitions currently.
We're at a loss for how to both debug and fix this. Restarting the Zookeeper
followers presumably will not help? Are all nodes ever force-synced from the
leader?
Help would be appreciated.
Thanks
Summary: Zookeeper followers not reflecting writes (after months)
(was: Zookeeper follower's not reflecting writes (after months))
> Zookeeper followers not reflecting writes (after months)
> --------------------------------------------------------
>
> Key: ZOOKEEPER-3490
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3490
> Project: ZooKeeper
> Issue Type: Bug
> Affects Versions: 3.4.13
> Reporter: Matthew Hertz
> Priority: Major
>
> Hi,
> We have a 3 node Zookeeper cluster. There are a number of znode's on the
> leader that are not visible on the followers.
> {code:java}
> $ zkCli -server <server 1> (follower)
> [zk: <server 1>(CONNECTED) 0] get
> /pyMkdProducer/SNAP/lock/c4a62c9fdfdc412fac3818bbb2af3a0f__lock__0000000040
> abcd.company.com:<built-in function getpid>
> cZxid = 0xf00061d68
> ctime = Thu Nov 01 12:40:33 GMT 2018
> mZxid = 0xf00061d68
> mtime = Thu Nov 01 12:40:33 GMT 2018
> pZxid = 0xf00061d68
> cversion = 0
> dataVersion = 0
> aclVersion = 0
> ephemeralOwner = 0x500be5318d60407
> dataLength = 58
> numChildren = 0
> {code}
> {code:java}
> $ zkCli -server <server 2> (leader)
> [zk: <server2>(CONNECTED) 0] get
> /pyMkdProducer/SNAP/lock/c4a62c9fdfdc412fac3818bbb2af3a0f__lock__0000000040
> Node does not exist:
> /pyMkdProducer/SNAP/lock/c4a62c9fdfdc412fac3818bbb2af3a0f__lock__0000000040
> {code}
> {code:java}
> $ zkCli -server <server 3> (follower)
> [zk: <server3>(CONNECTED) 0] get
> /pyMkdProducer/SNAP/lock/c4a62c9fdfdc412fac3818bbb2af3a0f__lock__0000000040
> abcd.company.com:<built-in function getpid>
> cZxid = 0xf00061d68
> ctime = Thu Nov 01 12:40:33 GMT 2018
> mZxid = 0xf00061d68
> mtime = Thu Nov 01 12:40:33 GMT 2018
> pZxid = 0xf00061d68
> cversion = 0
> dataVersion = 0
> aclVersion = 0
> ephemeralOwner = 0x500be5318d60407
> dataLength = 58
> numChildren = 0
> {code}
> These nodes are ephemeral nodes. The sessions no longer exist. There are 6
> znodes in this 'inconsistent' state. The cluster is currently connected -
> there are no networking partitions currently.
> We're at a loss for how to both debug and fix this. Restarting the Zookeeper
> followers presumably will not help? Are all nodes ever force-synced from the
> leader?
> Help would be appreciated. If any more information would be helpful it can be
> provided, however we will likely have to resolve this issue one way or
> another in the near future.
> Thanks
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
