Davis-Zhang-Onehouse commented on code in PR #13695:
URL: https://github.com/apache/hudi/pull/13695#discussion_r2285674558
##########
hudi-common/src/main/java/org/apache/hudi/common/util/HoodieDataUtils.java:
##########
@@ -94,4 +96,76 @@ public static <K, V> Map<K, Set<V>>
collectPairDataAsMap(HoodiePairData<K, V> pa
},
HashMap::putAll);
}
+
+ /**
+ * Executes a function with HoodieData and ensures cleanup after use
Review Comment:
HoodieData is an abstraction and its clean method also does not come with
any implementation. If we need to explain `cleanup` it means involve child
class specific implementation details.
The point I'm trying to make here is like "hoodie data is an auto closable,
make sure you call close once you are done with it". I doubt we need to explain
what `close` would do as auto closable is merely an interface that requires
some standard way of using it.
I can add hoodie RDD data clean up implementation details here - the clean
up method would call rdd.unpersist for the direct/indirect rdd dependency of
the current hoodie java rdd. lmk your thoughts. thank you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
