Vladimir Ivić created ZOOKEEPER-3594:
----------------------------------------
Summary: Ability to skip proposing requests with error transactions
Key: ZOOKEEPER-3594
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3594
Project: ZooKeeper
Issue Type: New Feature
Components: quorum, server
Affects Versions: 3.5.6
Reporter: Vladimir Ivić
Assignee: Vladimir Ivić
Ensembles that have a high write request rate could be skipping proposing
requests that contain errors instead of having the quorum vote on those
transactions.
For example, having a sizable write traffic that results in error transactions
would be creating additional network and log disk space overhead for each of
the requests that would only return errors to the client in the end (e.g.
delete non-existing path, version mismatch, trying to create a node that
already exists etc).
Currently, there is no such logic in the ProposalRequestProcessor, every
request that comes out of PrepRequestProcessor will be proposed to the quorum.
Proposed solution workflow:
* A client sends a new write request trying to setData on an non-existing node
* The server accepts the request and sends it through the PrepRequestProcessor
* PrepRequestProcessor detects the error and assigns the error transaction to
the request
* Between PrepRequestProcessor and ProposalRequestProcessor there is another
processor named SkipRequestProcessor which sole responsibility is to decide
will the request be forwarded or returned to the originating quorum peer
(Leader or Learner).
* The quorum peer waits for all previous requests to complete before the error
request proceeds with echoing the error back to the client.
Requirements:
* We should be conservative about the use of ZXID. If the request generates an
error transaction we don't want to increment the last proposed ZXID and cause
any gaps in the log.
* Request that are found to be invalid should be sent directly to the origin
(either to the corresponding Learner or to the Leader directly) so there is
exactly one roundtrip for each request with error transaction.
* All the requests must preserve its order, the changes must be backwards
compatible with the Zookeeper ordering guarantees.
Challenges:
* Skipping request without having them go through the proposal pipeline poses
a challenge in preserving Zookeeper transaction order.
* Avoiding any additional changes inside CommitProcessor if possible.
* Have unified logic for all three different paths in which write requests can
come through:
## Via Leader placed directly into the PrepRequestProcessor
## Via Follower where the request is forwarded to the leader and also passed
to CommitProcessor to wait for COMMIT packet
## Via Observer, where the request is forwarded to the Leader and the Observer
waits for the INFORM packet
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
