[
https://issues.apache.org/jira/browse/HADOOP-15392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16442927#comment-16442927
]
Sean Mackrory commented on HADOOP-15392:
----------------------------------------
{quote}If there's no way to stop this happening them the metric wire-up has to
be optional; keep off by default.{quote}
+1 to that approach if it comes to it. My first thought is that the metrics
configuration should be known at the time we make that decision, and in theory
we could only wire it up if the s3a metrics source would ever actually connect
to any sink. I suspect metrics2 doesn't expose the APIs to enable us to do that
in a clean way, though. I would also think that because it's designed for
daemons, the behavior in the event no sinks are configured would not be to just
leak memory forever, so I wonder if there's something else we're doing that's
wrong that we can just fix.
> S3A Metrics in S3AInstrumentation Cause Memory Leaks
> ----------------------------------------------------
>
> Key: HADOOP-15392
> URL: https://issues.apache.org/jira/browse/HADOOP-15392
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.1.0
> Reporter: Voyta
> Priority: Major
>
> While using HBase S3A Export Snapshot utility we started to experience memory
> leaks of the process after version upgrade.
> By running code analysis we traced the cause to revision
> 6555af81a26b0b72ec3bee7034e01f5bd84b1564 that added the following static
> reference (singleton):
> private static MetricsSystem metricsSystem = null;
> When application uses S3AFileSystem instance that is not closed immediately
> metrics are accumulated in this instance and memory grows without any limit.
>
> Expectation:
> * It would be nice to have an option to disable metrics completely as this
> is not needed for Export Snapshot utility.
> * Usage of S3AFileSystem should not contain any static object that can grow
> indefinitely.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
