alexeykudinkin commented on code in PR #6815:
URL: https://github.com/apache/hudi/pull/6815#discussion_r1083233569
##########
hudi-common/src/main/java/org/apache/hudi/common/fs/HoodieWrapperFileSystem.java:
##########
@@ -1064,4 +1057,22 @@ public void createImmutableFileInPath(Path fullPath,
Option<byte[]> content)
public FileSystem getFileSystem() {
return fileSystem;
}
+
+ private static FSDataInputStream wrapLeakTrackingStream(FSDataInputStream
fsDataInputStream) throws IOException {
+ if (fsDataInputStream instanceof LeakTrackingFSDataInputStream) {
+ return fsDataInputStream;
+ }
+ return new LeakTrackingFSDataInputStream(fsDataInputStream);
+ }
+
+ private static FSDataInputStream wrapTimeTrackingStream(final Path path,
FSDataInputStream fsDataInputStream) throws IOException {
+ if (fsDataInputStream instanceof TimedFSDataInputStream) {
+ return fsDataInputStream;
+ }
+ return new TimedFSDataInputStream(path, fsDataInputStream);
+ }
+
+ private static FSDataInputStream wrapStream(final Path path,
FSDataInputStream fsDataInputStream) throws IOException {
+ return wrapLeakTrackingStream(wrapTimeTrackingStream(path,
fsDataInputStream));
Review Comment:
Good point w/ diagnostic mode!
In terms of `LeakTrackingFSDataInputStream` itself it does 2 things:
1. Passively tracks leaks (taking stacktrace when created) by logging
warnings when object is finalized before closed
2. Does best-effort attempts to reclaim leaked streams
As such i think there's value in keeping it always on
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
