[
https://issues.apache.org/jira/browse/YARN-11397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17648770#comment-17648770
]
Chris Nauroth commented on YARN-11397:
--------------------------------------
Is this the same {{S3AInstrumentation}} leak issue as HADOOP-18526, which is
scheduled for inclusion in the upcoming 3.3.5 release?
CC: [[email protected]]
> Memory leak when reading aggregated logs from s3
> (LogAggregationTFileController::readAggregatedLogs)
> ----------------------------------------------------------------------------------------------------
>
> Key: YARN-11397
> URL: https://issues.apache.org/jira/browse/YARN-11397
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation
> Affects Versions: 3.2.2
> Environment: Remote logs dir on s3.
> Reporter: Maciej Smolenski
> Priority: Critical
> Attachments: YarnLogsS3Issue.scala
>
>
> Reproduction code in the attachment.
> When collecting aggregated logs from s3 in a loop (see reproduction code) we
> can easily see that the number of 'S3AInstrumentation' is increasing although
> the number of 'S3AFileSystem' is not increasing. It means that
> 'S3AInstrumentation' is not released together with 'S3AFileSystem' as it
> should be. The root cause of this seems to be the missing close on
> S3AFileSystem.
> The issue seems similar to https://issues.apache.org/jira/browse/YARN-11039
> but the issue is a 'memory leak' (not a 'thread leak') and affected version
> is earlier here (3.2.2).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
