codope commented on code in PR #18988:
URL: https://github.com/apache/hudi/pull/18988#discussion_r3671589018
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/callback/common/HoodieWriteCommitCallbackMessage.java:
##########
@@ -69,10 +72,106 @@ public class HoodieWriteCommitCallbackMessage implements
Serializable {
*/
private final Option<Map<String, String>> extraMetadata;
+ /**
+ * Previous base file paths keyed by fileId, resolved lazily. Populated by
the write
+ * client from the cached FileSystemView so that callback implementations
don't have to
+ * rebuild a view. Empty for inserts and for callers that don't pre-resolve.
+ *
+ * <p>Resolution is deferred to the {@code prevFilePathsSupplier} and only
triggered on the
+ * first {@link #getPrevFilePaths()} call: a callback that never reads the
previous paths
+ * pays nothing (no FileSystemView access). The resolved value is memoized
here and is the
+ * form that gets serialized (see {@link #writeObject}).
+ */
+ private Map<String, PrevFilePaths> prevFilePaths;
+
+ /**
+ * Lazily resolves {@link #prevFilePaths}. Transient because it may capture a
+ * FileSystemView (not serializable); excluded from the generated getters so
it never
+ * leaks into JSON/Java serialization. The resolved map is memoized into
+ * {@link #prevFilePaths}.
+ */
+ @Getter(AccessLevel.NONE)
+ private final transient Supplier<Map<String, PrevFilePaths>>
prevFilePathsSupplier;
Review Comment:
largely agree but `Lazy` is not serializable, so the field is `transient`
and `writeObject`/`readObject` carry the resolved map across explicitly. Net
effect is unchanged from before. The getter still returns `Map`, not `Lazy`, so
the Jackson path in `convertToJsonString` (used by the built-in
HTTP/Kafka/Pulsar callbacks) emits the same JSON as before.
Added
`TestHoodieWriteCommitCallbackMessage#prevFilePathsSurviveJavaSerialization` and
`#unresolvedPrevFilePathsSerializeAsEmpty` to test the sede behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
