hudi-agent commented on code in PR #19837:
URL: https://github.com/apache/hudi/pull/19837#discussion_r3927515703
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/partitioner/DynamicBucketAssignFunction.java:
##########
@@ -143,6 +153,15 @@ public void processElement(HoodieFlinkInternalRow record,
Context ctx, Collector
out.collect(record);
}
+ /**
+ * Applies a preloaded index record emitted by a bootstrap operator directly
to the partitioned
+ * index backend, since its {@code recordKey -> fileGroupId} mapping is
already known and does not
+ * need bucket assignment. The record carries no row data, so it is never
emitted downstream.
+ */
+ private void processIndexRecord(HoodieFlinkInternalRow record) {
Review Comment:
🤖 `update()` here calls `getOrBootstrapPartition()`, which for the first
record of a partition triggers a full lazy bootstrap of that partition from the
MDT (`readRecordIndexLocations`). Since the whole point of the preloaded index
records is to warm the cache and avoid that MDT scan, doesn't this defeat the
purpose (and effectively load each partition twice)? Is the lazy bootstrap
expected to be skipped when preload is active, or is the redundancy intentional?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
