[
https://issues.apache.org/jira/browse/HADOOP-10840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14063933#comment-14063933
]
Chris Nauroth commented on HADOOP-10840:
----------------------------------------
Hi, [~shanyu]. I'm seeing some test failures when I run the tests under
hadoop-tools/hadoop-azure. Could you please investigate? Here is a stack
trace from one example:
{code}
Running org.apache.hadoop.fs.azure.TestNativeAzureFileSystemFileNameCheck
Tests run: 4, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.871 sec <<<
FAILURE! - in org.apache.hadoop.fs.azure.TestNativeAzureFileSystemFileNameCheck
testMkdirs(org.apache.hadoop.fs.azure.TestNativeAzureFileSystemFileNameCheck)
Time elapsed: 0.04 sec <<< ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.fs.azure.metrics.AzureFileSystemMetricsSystem.unregisterSource(AzureFileSystemMetricsSystem.java:65)
at
org.apache.hadoop.fs.azure.NativeAzureFileSystem.close(NativeAzureFileSystem.java:1352)
at
org.apache.hadoop.fs.azure.AzureBlobStorageTestAccount.cleanup(AzureBlobStorageTestAccount.java:745)
at
org.apache.hadoop.fs.azure.TestNativeAzureFileSystemFileNameCheck.tearDown(TestNativeAzureFileSystemFileNameCheck.java:55)
{code}
> Fix OutOfMemoryError caused by metrics system in Azure File System
> ------------------------------------------------------------------
>
> Key: HADOOP-10840
> URL: https://issues.apache.org/jira/browse/HADOOP-10840
> Project: Hadoop Common
> Issue Type: Bug
> Components: metrics
> Affects Versions: 2.4.1
> Reporter: shanyu zhao
> Assignee: shanyu zhao
> Attachments: HADOOP-10840.patch
>
>
> In Hadoop 2.x the Hadoop File System framework changed and no cache is
> implemented (refer to HADOOP-6356). This means for every WASB access, a new
> NativeAzureFileSystem is created, along which a Metrics source created and
> added to MetricsSystemImpl. Over time the sources accumulated, eating memory
> and causing Java OutOfMemoryError.
> The fix is to utilize the unregisterSource() method added to MetricsSystem in
> HADOOP-10839.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
