nsivabalan commented on code in PR #6815:
URL: https://github.com/apache/hudi/pull/6815#discussion_r1083236083
##########
hudi-common/src/main/java/org/apache/hudi/common/fs/HoodieWrapperFileSystem.java:
##########
@@ -1064,4 +1057,22 @@ public void createImmutableFileInPath(Path fullPath,
Option<byte[]> content)
public FileSystem getFileSystem() {
return fileSystem;
}
+
+ private static FSDataInputStream wrapLeakTrackingStream(FSDataInputStream
fsDataInputStream) throws IOException {
+ if (fsDataInputStream instanceof LeakTrackingFSDataInputStream) {
+ return fsDataInputStream;
+ }
+ return new LeakTrackingFSDataInputStream(fsDataInputStream);
+ }
+
+ private static FSDataInputStream wrapTimeTrackingStream(final Path path,
FSDataInputStream fsDataInputStream) throws IOException {
+ if (fsDataInputStream instanceof TimedFSDataInputStream) {
+ return fsDataInputStream;
+ }
+ return new TimedFSDataInputStream(path, fsDataInputStream);
+ }
+
+ private static FSDataInputStream wrapStream(final Path path,
FSDataInputStream fsDataInputStream) throws IOException {
+ return wrapLeakTrackingStream(wrapTimeTrackingStream(path,
fsDataInputStream));
Review Comment:
as I mentioned, I do see the value. not denying it. may be we can do this.
enable this leak tracking only if logging level if INFO or more. if its just
warn, we can disable the leak tracking. Ideally, I prefer to guard it using a
flag and so if someone rans into any issue, they can enable it for further runs
and get some insights.
also, with the diagnostic lens, we can also think about tracking reader
writers, iterators, and much more. So, if we start tracking one(like
leaktracking), other devs might start to add more such tracking w/o any guards.
So, trying to be cautious in setting a precedence.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
