[
https://issues.apache.org/jira/browse/ZOOKEEPER-4681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
krystal he updated ZOOKEEPER-4681:
----------------------------------
Description:
Using a [tool|https://github.com/kry4tall/CC-ZOO358] that I modifyed from
[Filip Niksic's zootester|https://github.com/fniksic/zootester] for testing
ZooKeeper, I discovered the following scenario which causes uncommitted
requests to be executed.
Zab protocol has three rounds: PROPOSE, ACK, and COMMIT. By adding relevant
code to the zookeeper source code，my tool can drop the PROPOSAL, ACK and COMMIT
messages and collect the values of some variables of each server at the end of
each round. Except affecting message reception, my code will not affect other
actions of Zookeeper.
Setup:
Replace directory called "zookeeper-server" in Zookeeper 3.5.8 with the
"zookeeper-server" in [my github repo|https://github.com/kry4tall/CC-ZOO358].
Ant the modified Zookeeper 3.5.8 to get zookeeper-3.5.8.jar. Replace
zookeeper-3.5.8.jar downloaded by maven.
Create a directory called "states" and a file called
"[scenarios|https://github.com/kry4tall/CC-ZOO358/blob/krystal/zoo-tester/test/scenarios]";.
Write the path to test.properties in zoo-tester's resource directory.
Use "-s divergence" or "-s divergence-X"(X = 2,3,4,5,6) as the startup
parameter to run the main method of ZooTester.
OS: ubuntu 22.04.2.
Base scenario:
Initially, start an ensemble with 3 servers called A, B, and C, and initialize
2 znodes called /key0 and /key1, and set them to 0 and 1 respectively.
# Request to set /key0 to 1000 on 3 servers.
# Isolate the ack messages of 2 followers.
# (Optional) Stop all servers and then restart them.
# (Optional) Read /key0 and /key1.
# Request to set /key0 to 1001 on 3 servers.
# (Optional) Stop all servers and then restart them.
# Read /key0 and /key1.
Mark the execution step list [1,2,4,5,7] as scenario1, [1,2,3,4,5,7] as
scenario2, [1,2,4,5,6,7] as scenario3, [1,2,3,4,5,6,7] as scenario4, [1,2,5,7]
as scenario5 and [1,2,5,6,7] as scenario6.
The output of these 6 scenarios is placed in the attachment. Surprisingly, key0
is set to 1000 and key1 is set to 1001 in some scenarios and
However, value 1000 and 1001 should not appear in any znode, because the
proposal of the first request cannot obtain enough ack, it cannot be committed.
And servers will also not commit the proposal of the second request, because
there is a pending proposal that has not been committed before, according to
the source code of Zookeeper.
was:
Using a [tool|https://github.com/kry4tall/CC-ZOO358] that I modifyed from
[Filip Niksic's zootester|https://github.com/fniksic/zootester] for testing
ZooKeeper, I discovered the following scenario which causes uncommitted
requests to be executed.
Zab protocol has three rounds: PROPOSE, ACK, and COMMIT. By adding relevant
code to the zookeeper source code，my tool can drop the PROPOSAL, ACK and COMMIT
messages and collect the values of some variables of each server at the end of
each round. Except affecting message reception, my code will not affect other
actions of Zookeeper.
Setup:
Replace directory called "zookeeper-server" in Zookeeper 3.5.8 with the
"zookeeper-server" in [my github repo|https://github.com/kry4tall/CC-ZOO358].
Ant the modified Zookeeper 3.5.8 to get zookeeper-3.5.8.jar. Replace
zookeeper-3.5.8.jar downloaded by maven.
Create a directory called "states" and a file called
"[scenarios|https://github.com/kry4tall/CC-ZOO358/blob/krystal/zoo-tester/test/scenarios]";.
Write the path to test.properties in zoo-tester's resource directory.
Use "-s divergence" or "-s divergence-X"(X = 2,3,4,5,6) as the startup
parameter to run the main method of ZooTester.
OS: ubuntu 22.04.2.
Base scenario:
Initially, start an ensemble with 3 servers called A, B, and C, and initialize
2 znodes called /key0 and /key1, and set them to 0 and 1 respectively.
# Request to set /key0 to 1000 on 3 servers.
# Isolate the ack messages of 2 followers.
# (Optional) Stop all servers and then restart them.
# (Optional) Read /key0 and /key1.
# Request to set /key0 to 1001 on 3 servers.
# (Optional) Stop all servers and then restart them.
# Read /key0 and /key1.
Mark the execution step list [1,2,4,5,7] as scenario1, [1,2,3,4,5,7] as
scenario2, [1,2,4,5,6,7] as scenario3, [1,2,3,4,5,6,7] as scenario4, [1,2,5,7]
as scenario5 and [1,2,5,6,7] as scenario6.
The output of these 6 scenarios is placed in the attachment. Surprisingly, key0
is set to 1000 and key1 is set to 1001 in some scenarios.
However, value 1000 and 1001 should not appear in any znode, because the
proposal of the first request cannot obtain enough ack, it cannot be committed.
And servers will also not commit the proposal of the second request, because
there is a pending proposal that has not been committed before, according to
the source code of Zookeeper.
> Uncommitted requests have been executed
> ----------------------------------------
>
> Key: ZOOKEEPER-4681
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4681
> Project: ZooKeeper
> Issue Type: Bug
> Components: quorum
> Affects Versions: 3.5.8
> Reporter: krystal he
> Priority: Critical
> Attachments: zookeeper-scenario1.patch, zookeeper-scenario2.patch,
> zookeeper-scenario3.patch, zookeeper-scenario4.patch,
> zookeeper-scenario5.patch, zookeeper-scenario6.patch
>
>
> Using a [tool|https://github.com/kry4tall/CC-ZOO358] that I modifyed from
> [Filip Niksic's zootester|https://github.com/fniksic/zootester] for testing
> ZooKeeper, I discovered the following scenario which causes uncommitted
> requests to be executed.
> Zab protocol has three rounds: PROPOSE, ACK, and COMMIT. By adding relevant
> code to the zookeeper source code，my tool can drop the PROPOSAL, ACK and
> COMMIT messages and collect the values of some variables of each server at
> the end of each round. Except affecting message reception, my code will not
> affect other actions of Zookeeper.
>
> Setup:
> Replace directory called "zookeeper-server" in Zookeeper 3.5.8 with the
> "zookeeper-server" in [my github repo|https://github.com/kry4tall/CC-ZOO358].
> Ant the modified Zookeeper 3.5.8 to get zookeeper-3.5.8.jar. Replace
> zookeeper-3.5.8.jar downloaded by maven.
> Create a directory called "states" and a file called
> "[scenarios|https://github.com/kry4tall/CC-ZOO358/blob/krystal/zoo-tester/test/scenarios]";.
> Write the path to test.properties in zoo-tester's resource directory.
> Use "-s divergence" or "-s divergence-X"(X = 2,3,4,5,6) as the startup
> parameter to run the main method of ZooTester.
> OS: ubuntu 22.04.2.
>
> Base scenario:
> Initially, start an ensemble with 3 servers called A, B, and C, and
> initialize 2 znodes called /key0 and /key1, and set them to 0 and 1
> respectively.
> # Request to set /key0 to 1000 on 3 servers.
> # Isolate the ack messages of 2 followers.
> # (Optional) Stop all servers and then restart them.
> # (Optional) Read /key0 and /key1.
> # Request to set /key0 to 1001 on 3 servers.
> # (Optional) Stop all servers and then restart them.
> # Read /key0 and /key1.
> Mark the execution step list [1,2,4,5,7] as scenario1, [1,2,3,4,5,7] as
> scenario2, [1,2,4,5,6,7] as scenario3, [1,2,3,4,5,6,7] as scenario4,
> [1,2,5,7] as scenario5 and [1,2,5,6,7] as scenario6.
> The output of these 6 scenarios is placed in the attachment. Surprisingly,
> key0 is set to 1000 and key1 is set to 1001 in some scenarios and
> However, value 1000 and 1001 should not appear in any znode, because the
> proposal of the first request cannot obtain enough ack, it cannot be
> committed. And servers will also not commit the proposal of the second
> request, because there is a pending proposal that has not been committed
> before, according to the source code of Zookeeper.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
