mumrah commented on code in PR #16862:
URL: https://github.com/apache/kafka/pull/16862#discussion_r1715321880
##########
raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java:
##########
@@ -3302,19 +3297,34 @@ private long append(int epoch, List<T> records,
OptionalLong requiredBaseOffset,
BatchAccumulator<T> accumulator = leaderState.accumulator();
boolean isFirstAppend = accumulator.isEmpty();
- final long offset = accumulator.append(epoch, records,
requiredBaseOffset, isAtomic);
+ final long offset = accumulator.append(epoch, records, true);
Review Comment:
Just curious, did we ever do non-atomic appends in the Raft layer before? I
think the controller only did atomic appends (even though the controller itself
would split up non-atomic batches)
##########
metadata/src/main/java/org/apache/kafka/controller/QuorumController.java:
##########
@@ -817,27 +817,25 @@ public void run() throws Exception {
// succeed; if it does not, that's a fatal error. It
is important to do this before
// scheduling the record for Raft replication.
int recordIndex = 0;
- long nextWriteOffset = offsetControl.nextWriteOffset();
+ long lastOffset =
raftClient.scheduleAppend(controllerEpoch, records);
+ long baseOffset = lastOffset - records.size() + 1;
for (ApiMessageAndVersion message : records) {
- long recordOffset = nextWriteOffset + recordIndex;
+ long recordOffset = baseOffset + recordIndex;
try {
replay(message.message(), Optional.empty(),
recordOffset);
} catch (Throwable e) {
String failureMessage = String.format("Unable
to apply %s " +
"record at offset %d on active controller,
from the " +
"batch with baseOffset %d",
message.message().getClass().getSimpleName(),
- recordOffset, nextWriteOffset);
+ recordOffset, baseOffset);
throw
fatalFaultHandler.handleFault(failureMessage, e);
}
recordIndex++;
}
- long nextEndOffset = nextWriteOffset - 1 + recordIndex;
- raftClient.scheduleAtomicAppend(controllerEpoch,
- OptionalLong.of(nextWriteOffset),
- records);
-
offsetControl.handleScheduleAtomicAppend(nextEndOffset);
- return nextEndOffset;
+ raftClient.scheduleFlush();
Review Comment:
Any side effect if the batch already flushed by the time we get here? I
suspect not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
