szetszwo commented on PR #1362: URL: https://github.com/apache/ratis/pull/1362#issuecomment-3986069136
> How about we simply update the `repliedIndex` to the current leader's `appliedIndex` in `ReplyFlusher#flush`? ... @ivandika3 , Even `repliedIndex` is updated with `appliedIndex`, [the same problem](https://github.com/apache/ratis/pull/1362#discussion_r2870030074) can happen. Just found that the problem can happen using `appliedIndex` as the `readIndex`: 1. Leader: committedIndex=20 and appliedIndex=10 since the entries are not yet applied due to slowness. 2. Client first reads from F1: F1 gets readIndex = 10 but its appliedIndex is 18. So, it replies at appliedIndex = 18 3. The same client reads from F2: It gets readIndex = 10 and replies at appliedIndex = 14 The original readIndex algorithm has the invariant: - Original readIndex algorithm: follower's appliedIndex <= follower's committedIndex <= leader's committedIndex When a follower receives a read request, it cannot get a readIndex (= leader's committedIndex) > its appliedIndex at that time. There are two options to fix it: 1. Change Leader to send appliedIndex (or repliedIndex depending on the readIndex conf) as [leaderCommit](https://github.com/apache/ratis/blob/a5651f6e9a95f5b5ac44ff01363b7de1c8b4c026/ratis-proto/src/main/proto/Raft.proto#L185) in appendEntries. Then, the following invariants will hold: - Using appliedIndex as readIndex: follower's appliedIndex <= follower's committedIndex <= leader's appliedIndex - Using repliedIndex as readIndex: follower's appliedIndex <= follower's committedIndex <= leader's repliedIndex 2. Add a seenIndex to the read requests. The server (leader or follower) will wait max(seenIndex, readIndex) is applied. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
