[
https://issues.apache.org/jira/browse/HDFS-13234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16390298#comment-16390298
]
Kihwal Lee commented on HDFS-13234:
-----------------------------------
New conf objects are created to prevent unintended conf update propagation. If
we have an overlay config feature, we could achieve the same thing without
duplicating the entire conf object. Configuration has something overlay-like,
but I was told it does not work the way we want.
> Remove renew configuration instance in ConfiguredFailoverProxyProvider and
> reduce memory footprint for client
> -------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-13234
> URL: https://issues.apache.org/jira/browse/HDFS-13234
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: fs, ha, hdfs-client
> Reporter: He Xiaoqiao
> Priority: Major
> Attachments: HDFS-13234.001.patch
>
>
> The memory footprint of #DFSClient is very considerable in some special
> scenario since there are many #Configuration instances and occupy much memory
> resource (In an extreme case, org.apache.hadoop.conf.Configuration occupies
> over 600MB we meet under HDFS Federation an HA with QJM and there are dozens
> of NameNodes). I think some new Configuration instance is not necessary. Such
> as #ConfiguredFailoverProxyProvider initialization.
> {code:java}
> public ConfiguredFailoverProxyProvider(Configuration conf, URI uri,
> Class<T> xface, HAProxyFactory<T> factory) {
> this.xface = xface;
> this.conf = new Configuration(conf);
> ......
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
