[
https://issues.apache.org/jira/browse/HADOOP-10840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shanyu zhao updated HADOOP-10840:
---------------------------------
Attachment: HADOOP-10840.patch
Patch attached.
This patch added a finalize() method to NativeAzureFileSystem to force a call
to close() which calls unregisterSoruce() on the metrics system impl. Note that
close() can be called multiple times without negative impact so that in the
future when we introduce close() to FileContext to let client manually call
close() there shall be no changes in this area. I also removed registration
code to DefaultMetricsSystem because we only need to register with one metrics
system implementation which is AzureFileSystemMetricsSystem.
> Fix OutOfMemoryError caused by metrics system in Azure File System
> ------------------------------------------------------------------
>
> Key: HADOOP-10840
> URL: https://issues.apache.org/jira/browse/HADOOP-10840
> Project: Hadoop Common
> Issue Type: Bug
> Components: metrics
> Affects Versions: 2.4.1
> Reporter: shanyu zhao
> Assignee: shanyu zhao
> Attachments: HADOOP-10840.patch
>
>
> In Hadoop 2.x the Hadoop File System framework changed and no cache is
> implemented (refer to HADOOP-6356). This means for every WASB access, a new
> NativeAzureFileSystem is created, along which a Metrics source created and
> added to MetricsSystemImpl. Over time the sources accumulated, eating memory
> and causing Java OutOfMemoryError.
> The fix is to utilize the unregisterSource() method added to MetricsSystem in
> HADOOP-10839.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
