mumrah commented on code in PR #17398:
URL: https://github.com/apache/kafka/pull/17398#discussion_r1793541831
##########
metadata/src/main/java/org/apache/kafka/image/publisher/SnapshotGenerator.java:
##########
@@ -240,7 +240,11 @@ void publishLogDelta(
LogDeltaManifest manifest
) {
bytesSinceLastSnapshot += manifest.numBytes();
- if (bytesSinceLastSnapshot >= maxBytesSinceLastSnapshot) {
+ if (!manifest.provenance().isOffsetBatchAligned()) {
+ if (log.isTraceEnabled()) {
+ log.trace("Not scheduling snapshot because last contained
offset is not aligned to a batch boundary.");
Review Comment:
This will cause a trace log on every metadata publish regardless of the
`bytesSinceLastSnapshot` or other snapshot conditions being met.
I'd suggest pushing this down into the existing branches below and check
`isOffsetBatchAligned` along with `eventQueue.isEmpty()`
Also, since this scenario will probably be rare, I'd suggest DEBUG level
##########
metadata/src/main/java/org/apache/kafka/image/loader/MetadataBatchLoader.java:
##########
@@ -143,7 +143,8 @@ public long loadBatch(Batch<ApiMessageAndVersion> batch,
LeaderAndEpoch leaderAn
// 1) this is not the first record in this batch
// 2) this is not the first batch since last emitting a delta
if (transactionState == TransactionState.STARTED_TRANSACTION &&
(indexWithinBatch > 0 || numBatches > 0)) {
- MetadataProvenance provenance = new
MetadataProvenance(lastOffset, lastEpoch, lastContainedLogTimeMs);
+ // Accumulated delta is aligned on batch boundaries iff the
BeginTransactionRecord is the first record in a batch
+ MetadataProvenance provenance = new
MetadataProvenance(lastOffset, lastEpoch, lastContainedLogTimeMs,
indexWithinBatch == 0);
Review Comment:
We should update the debug log a few lines below this.
From
> handleCommit: Generated a metadata delta between 100 and 120 from 2
batch(es) in 4 us
To
> handleCommit: Generated an unaligned metadata delta between 100 and 120
from 2 batch(es) in 4 us
or similar
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
