bitflicker64 commented on code in PR #3376:
URL: https://github.com/apache/hugegraph/pull/3376#discussion_r4326263396
##########
hugegraph-pd/hg-pd-core/src/main/java/org/apache/hugegraph/pd/store/RaftKVStore.java:
##########
@@ -329,6 +393,25 @@ private boolean isLeader() {
@Override
public boolean invoke(KVOperation op, KVStoreClosure response) throws
PDException {
+ if (this.appliedBefore(op)) {
+ if (response != null) {
+ /*
+ * Not a replay: this node proposed the entry in this run, and
the log gave it
+ * an index the store claims to hold already. The store and
the log do not
+ * match (a store restored without its log, or one written on
the path
+ * without raft by an older release); refused, since a skipped
write would
+ * answer success and write nothing.
+ */
+ long last = KvTxnApplier.lastIndex(this.store);
+ String message = "Refused raft entry " + op.getIndex() + ":
the store holds " +
+ "TXN index " + last + " already, so the store
and the raft " +
+ "log do not match";
+ log.error(message);
+
response.setError(Pdpb.Error.newBuilder().setType(Pdpb.ErrorType.UNKNOWN)
+ .setMessage(message).build());
+ }
+ return false;
Review Comment:
Fixed in 8f29ffdd4. The skip is now limited to replays: `RaftEngine` records
the log's last index
when the node starts, and `RaftKVStore.appliedBefore` skips only an entry at
or below it. A newer
entry the store already claims makes `invoke` throw on every node, caller or
not, and
`RaftStateMachine.onApply` turns that into `setErrorAndRollback`: jraft
stops the state machine,
a leader steps down, the probes from #3358 report the error, and the node
logs the entry, the
stored index and the recovery (remove the data directory, rejoin from a
snapshot) at error level.
The earlier closure-error path is gone. Test
`testStoreAheadOfTheLogFailsTheApply` covers a
replay, a follower and the proposer path; the API reference says what each
node does.
##########
hugegraph-pd/hg-pd-core/src/main/java/org/apache/hugegraph/pd/raft/KVOperation.java:
##########
@@ -49,6 +49,11 @@ public class KVOperation {
public static final byte CLEAR = 0x07;
public static final byte PUT_WITH_TTL_UNIT = 0x08;
public static final byte REMOVE_WITH_TTL = 0x09;
+ /**
+ * Atomic multi-key write, the value holds a serialized kv.TxnEntry: the
request with its
+ * keys encoded by the proposing node
+ */
+ public static final byte TXN = 0x0A;
Review Comment:
Fixed in 8f29ffdd4. The leader proposes a TXN only once every other voter
and learner of the
configuration has answered its member probe (`GetMemberResponse`, new field
`maxKvOp`) as a
release that applies the entry. `TxnPeerGate` keeps that answer for the life
of the process; a
peer that never answered, or answered as an older release, blocks the TXN
with
`TXN_NOT_PROPOSED` naming it, a confirmed peer that is unreachable later
does not block, and
unconfirmed peers are probed again at most every 3 s and at once on a
configuration change, so a
node added to the group blocks until it answers. `TxnPeerGateTest` covers
each rule; the API
reference and the description's rollout note state them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
