satishd commented on a change in pull request #11058:
URL: https://github.com/apache/kafka/pull/11058#discussion_r712001447
##########
File path:
storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/ConsumerTask.java
##########
@@ -120,6 +174,29 @@ public void run() {
}
}
+ private void syncCommittedDataAndOffsets(boolean forceSync) {
+ boolean noOffsetUpdates =
committedPartitionToConsumedOffsets.equals(partitionToConsumedOffsets);
+ if (noOffsetUpdates || !forceSync && time.milliseconds() -
lastSyncedTimeMs < committedOffsetSyncIntervalMs) {
+ log.debug("Skip syncing committed offsets, noOffsetUpdates: {},
forceSync: {}", noOffsetUpdates, forceSync);
+ return;
+ }
+
+ try {
+ // todo sync the snapshot file
+ for (TopicIdPartition topicIdPartition : assignedTopicPartitions) {
+ int metadataPartition =
topicPartitioner.metadataPartition(topicIdPartition);
+
remotePartitionMetadataEventHandler.syncLogMetadataDataFile(topicIdPartition,
metadataPartition,
+
partitionToConsumedOffsets.get(metadataPartition));
+ }
+
+ committedOffsetsFile.writeEntries(partitionToConsumedOffsets);
+ committedPartitionToConsumedOffsets = new
HashMap<>(partitionToConsumedOffsets);
+ lastSyncedTimeMs = time.milliseconds();
+ } catch (IOException e) {
+ log.error("Error encountered while writing committed offsets to a
local file", e);
Review comment:
For now, we will log an error if there are any issues in writing to the
file. There are no direct dependencies to `logDirFailureChannel` as it is part
of RLMM implementation. We can address this in PRs when remote log subsystem is
added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
