[
https://issues.apache.org/jira/browse/HBASE-20433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454717#comment-16454717
]
Sean Mackrory commented on HBASE-20433:
---------------------------------------
I debugged a run of ExportSnapshot in connection with HADOOP-15392. In my case
I see an FS instance get created for the snapshot directory (e.g.
s3a://my-bucket/hbase-snapshots) and another FS instance get created for the
specific snapshot (e.g.
s3a://my-bucket/hbase-snapshots/snapshot-name-8015e164-781d-4187-afa1-bcc25541ba9e).
The latter is the one where the FS instance is turned off. They also appear to
get closed and the metrics system stopped when second one is closed too. There
are ~100 files in the snapshot and I just got the 2 FS instances you'd expect,
so I don't see how a 53k-file snapshot is resulting in 53k FS instances in the
driver.
> HBase Export Snapshot utility does not close FileSystem instances
> -----------------------------------------------------------------
>
> Key: HBASE-20433
> URL: https://issues.apache.org/jira/browse/HBASE-20433
> Project: HBase
> Issue Type: Bug
> Components: Client, fs, snapshots
> Affects Versions: 1.2.6, 1.4.3
> Reporter: Voyta
> Priority: Major
>
> It seems org.apache.hadoop.hbase.snapshot.ExportSnapshot disallows FileSystem
> instance caching.
> When verifySnapshot method is being run it calls often methods like
> org.apache.hadoop.hbase.util.FSUtils#getRootDir that instantiate FileSystem
> but never calls org.apache.hadoop.fs.FileSystem#close method. This behaviour
> allows allocation of unwanted objects potentially causing memory leaks.
> Related issue: https://issues.apache.org/jira/browse/HADOOP-15392
>
> Expectation:
> * HBase should properly release/close all objects, especially FileSystem
> instances.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
