danny0405 commented on code in PR #17770:
URL: https://github.com/apache/hudi/pull/17770#discussion_r2670703394
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/ArchivedTimelineLoader.java:
##########
@@ -35,37 +34,43 @@ public interface ArchivedTimelineLoader extends
Serializable {
/**
* Loads the instants from the timeline.
*
- * @param metaClient The meta client.
- * @param filter The time range filter where the target instant
belongs to.
- * @param loadMode The load mode.
- * @param commitsFilter Filter of the instant type.
- * @param recordConsumer Consumer of the instant record payload.
+ * @param metaClient The meta client.
+ * @param fileTimeRangeFilter The time range filter where the target instant
belongs to. This will only be applied at the file level.
+ * If strict filtering is required for the
records, the commitsFilter should be used as well.
+ * @param loadMode The load mode.
+ * @param commitsFilter Filter of the instant type.
+ * @param recordConsumer Consumer of the instant record payload.
+ *
+ * @return The last instant time loaded or empty if no instant is loaded.
*/
- void loadInstants(
+ Option<String> loadInstants(
HoodieTableMetaClient metaClient,
- @Nullable HoodieArchivedTimeline.TimeRangeFilter filter,
+ @Nullable HoodieArchivedTimeline.TimeRangeFilter fileTimeRangeFilter,
HoodieArchivedTimeline.LoadMode loadMode,
Function<GenericRecord, Boolean> commitsFilter,
BiConsumer<String, GenericRecord> recordConsumer);
/**
* Loads the instants from the timeline with optional limit for early
termination.
*
- * @param metaClient The meta client.
- * @param filter The time range filter where the target instant
belongs to.
- * @param loadMode The load mode.
- * @param commitsFilter Filter of the instant type.
- * @param recordConsumer Consumer of the instant record payload.
- * @param limit Optional maximum number of instants to load. Empty
for no limit.
+ * @param metaClient The meta client.
+ * @param fileTimeRangeFilter The time range filter where the target instant
belongs to. This will only be applied at the file level.
+ * If strict filtering is required for the
records, the commitsFilter should be used as well.
+ * @param loadMode The load mode.
+ * @param commitsFilter Filter of the instant type.
+ * @param recordConsumer Consumer of the instant record payload.
Review Comment:
I see now all the impls moves the record instant filtering into the
`recordConsumer`, is this a concention that you want the developer to follow,
should we doc it well here then?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
