[
https://issues.apache.org/jira/browse/HADOOP-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-12490:
------------------------------------
Assignee: Matthias Jacob
Affects Version/s: 2.7.1
Status: Patch Available (was: Open)
submitting. It's not ready to go in, with the big issue being "what happens if
that XML file isn't a valid property file?". That's not something you are
testing for is it? What do you expect to happen?
> Add default resources to Configuration which are not inside the class path
> --------------------------------------------------------------------------
>
> Key: HADOOP-12490
> URL: https://issues.apache.org/jira/browse/HADOOP-12490
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Affects Versions: 2.7.1
> Reporter: Matthias Jacob
> Assignee: Matthias Jacob
> Attachments: HADOOP-12490.001.patch
>
>
> *Problem*
> It is currently not possible to add default resources to
> {{org.apache.hadoop.conf.Configuration}} that are not included in the class
> path.
> We need to add resources to {{org.apache.hadoop.conf.Configuration}} that are
> not included in the class path. Currently, the only way to do this is to use
> {{addResource(URL)}} or {{addResource(Path)}} which both are instance
> methods. However, a library we work with creates Configuration instances
> internally that we cannot access. So we need to add them as default resources
> in order to make them available to all subsequent Configuration instances
> (and those existing instances where {{loadDefaults}} equals {{true}}, of
> course).
> *Solution*
> The proposed solution is to add the following static methods:
> {code}
> public static synchronized void addDefaultResource(URL url);
> public static synchronized void addDefaultResource(Path file);
> {code}
> Please see the attached patch for details.
> *Example*
> In order to initialize the Configuration correctly, what we need to do
> currently is as follows:
> {code}
> // Set resource paths for the class loader
> File ffile = new File("/path/to/conf/dir");
> URL url = ffile.toURI().toURL();
> URLClassLoader systemClassLoader = (URLClassLoader)
> getClass().getClassLoader();
> Class<URLClassLoader> classLoaderClass = URLClassLoader.class;
> Method method = classLoaderClass.getDeclaredMethod("addURL", new
> Class[]{URL.class});
> method.setAccessible(true);
> method.invoke(systemClassLoader, new Object[]{url});
> //-----------------------------
> // Read cluster config
> for (File f : ffile.listFiles()) {
> if(f.getName().toLowerCase().endsWith(".xml")) {
> Configuration.addDefaultResource(f.getName());
> }
> }
> {code}
> As you can see, we currently need to manipulate the class path through
> reflection which is kind of dirty.
> With the proposed solution, the same (adding default resources from outside
> the class path) can be achieved doing the following:
> {code}
> // Set resource paths for the class loader
> File ffile = new File("/path/to/conf/dir");
> //-----------------------------
> // Read cluster config
> for (File f : ffile.listFiles()) {
> if(f.getName().toLowerCase().endsWith(".xml")) {
> Configuration.addDefaultResource(new Path(f.getAbsolutePath()));
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
