PavithranRick commented on code in PR #14261:
URL: https://github.com/apache/hudi/pull/14261#discussion_r2561933739
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/versioning/v2/ArchivedTimelineLoaderV2.java:
##########
@@ -56,29 +59,58 @@ public void loadInstants(HoodieTableMetaClient metaClient,
// List all files
List<String> fileNames = LSMTimeline.latestSnapshotManifest(metaClient,
metaClient.getArchivePath()).getFileNames();
+ // Check if consumer supports early termination
+ StoppableRecordConsumer stoppable = recordConsumer instanceof
StoppableRecordConsumer
+ ? (StoppableRecordConsumer) recordConsumer
+ : null;
+
+ // Filter files by time range
+ List<String> filteredFiles = new ArrayList<>();
+ for (String fileName : fileNames) {
+ if (filter == null || LSMTimeline.isFileInRange(filter, fileName)) {
+ filteredFiles.add(fileName);
+ }
+ }
+
+ // Sort files in reverse chronological order if needed (newest first for
limit queries)
+ if (stoppable != null && stoppable.needsReverseOrder()) {
Review Comment:
makes sense. Accepted the suggestion and made relevant changes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
