[
https://issues.apache.org/jira/browse/HADOOP-9570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daryn Sharp updated HADOOP-9570:
--------------------------------
Status: Patch Available (was: Open)
Re-discovered this exact issue due to performance issues caused by SSLFactory's
loading of ssl-client/server xmls triggering a full reload of configs. I was
about to submit a new Jira until [~jeagles] pointed me to this ancient one.
The only "risk" I can see is if something is creating a conf object,
duplicating it and adding a resource, and relying on changes in the other
resources to be reflected in the duplicated object. I'm inclined to think that
violates the natural semantics of expecting a copy to be a copy. Explicitly
reload the conf if that's the desired behavior. I also wonder but don't have
time to test how the former implicit reload worked with non-file resources and
whether this patch will be a benefit to that use case.
Don't have time for trunk port if someone would like to help.
> Configuration.addResource() should only parse the new resource
> --------------------------------------------------------------
>
> Key: HADOOP-9570
> URL: https://issues.apache.org/jira/browse/HADOOP-9570
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Environment: Ubuntu LXC
> Reporter: Gopal V
> Assignee: Daryn Sharp
> Priority: Minor
> Attachments: HADOOP-9570.branch-2.8.patch, HADOOP-9750.patch
>
>
> Hadoop configuration parsing re-parses all configuration files when a new
> resource is added to the configuration object.
> This is wasteful and runs through every deprecation check unnecessarily.
> {code}
> JobConf clone = new JobConf(job);
> clone.addResource(...);
> {code}
> will reparse every file including core-site, hdfs-site etc.
> Resource addition can be taken care of cleanly, if the addResourceObject()
> call applies the new resource, followed by applying the overlay, without
> re-parsing any of the older files already loaded into the hashtable.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
