[
https://issues.apache.org/jira/browse/HADOOP-15658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16579750#comment-16579750
]
Piotr Nowojski commented on HADOOP-15658:
-----------------------------------------
Without using {{deleteOnExit}} this local temporary file should also be
deleted/taken care of in case of exceptions, not only on happy paths.
When referring to HADOOP-8635 I wrote in "different component", where I only
meant that this "faulty" `deleteOnExit` call has been previously
acknowledged/recognised by Hadoop as problematic and that it should be avoided
(and popped up when I was searching for a known/previous issues). But yes, I
could be more clear that this old fix is unrelated to the current problem,
sorry about that.
And yes this issue is problematic for long living applications that also create
a lot of files (that's why even we didn't know about this problem before).
> Memory leak in S3AOutputStream
> ------------------------------
>
> Key: HADOOP-15658
> URL: https://issues.apache.org/jira/browse/HADOOP-15658
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
> Affects Versions: 2.8.4
> Reporter: Piotr Nowojski
> Priority: Major
>
> S3AOutputStream by calling
> {{org.apache.hadoop.fs.s3a.S3AFileSystem#createTmpFileForWrite}} indirectly
> calls {{java.io.File#deleteOnExit}} and \{{java.io.DeleteOnExitHook}} which
> are known for memory leaking:
>
> [https://bugs.java.com/view_bug.do?bug_id=6664633]
> [https://bugs.java.com/view_bug.do?bug_id=4872014]
>
> Apparently it was even fixed for different component couple of years ago
> https://issues.apache.org/jira/browse/HADOOP-8635 .
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
