[jira] [Commented] (HADOOP-17088) Failed to load Xinclude files with relative path in case of loading conf via URI
[
https://issues.apache.org/jira/browse/HADOOP-17088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17198824#comment-17198824
]
Yushi Hayasaka commented on HADOOP-17088:
-
Let me add an example.
Usually, when we want to load a Hadoop configuration file, we write like that:
* Main.java
{code:java}
package com.example.test;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import java.io.File;
public class Main {
public static void main(String[] args) {
String filePath = "/tmp/core-site.xml";
File file = new File(filePath);
Configuration configuration = new Configuration(false);
configuration.addResource(new Path(file.toURI()));
configuration.get("test");
}
}
{code}
* /tmp/core-site.xml
{code:xml}
http://www.w3.org/2001/XInclude;>
fs.defaultFS
viewfs://clusterX
{code}
* /tmp/mountTable.xml
{code:xml}
fs.viewfs.mounttable.ClusterX.link./data
hdfs://nn1-clusterx.example.com:8020/data
{code}
(I used [ViewFs
Guide|https://hadoop.apache.org/docs/r3.3.0/hadoop-project-dist/hadoop-hdfs/ViewFs.html]
example)
This example passes `Path` object to `addResource` parameter and the `Path` is
created from `URI` through `File.toURI()`. But actually it does not work
because of the error as I explain on the description of this ticket at first.
The example works well when we use `file.getPath()` (returns String) instead of
`toURI`.
Also, I think there is no difference with a security point of view because we
can load the XML file which is specified by `xi:include` and located outside of
the classpath when we load a configuration file using `file.getPath()`.
> Failed to load Xinclude files with relative path in case of loading conf via
> URI
>
>
> Key: HADOOP-17088
> URL: https://issues.apache.org/jira/browse/HADOOP-17088
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Yushi Hayasaka
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When we create a configuration file, which load a external XML file with
> relative path, and try to load it via calling `Configuration.addResource`
> with `Path(URI)`, we got an error, which failed to load a external XML, after
> https://issues.apache.org/jira/browse/HADOOP-14216 is merged.
> {noformat}
> Exception in thread "main" java.lang.RuntimeException: java.io.IOException:
> Fetch fail on include for 'mountTable.xml' with no fallback while loading
> 'file:/opt/hadoop/etc/hadoop/core-site.xml'
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3021)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2973)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2848)
> at
> org.apache.hadoop.conf.Configuration.iterator(Configuration.java:2896)
> at com.company.test.Main.main(Main.java:29)
> Caused by: java.io.IOException: Fetch fail on include for 'mountTable.xml'
> with no fallback while loading 'file:/opt/hadoop/etc/hadoop/core-site.xml'
> at
> org.apache.hadoop.conf.Configuration$Parser.handleEndElement(Configuration.java:3271)
> at
> org.apache.hadoop.conf.Configuration$Parser.parseNext(Configuration.java:3331)
> at
> org.apache.hadoop.conf.Configuration$Parser.parse(Configuration.java:3114)
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3007)
> ... 4 more
> {noformat}
> The cause is that the URI is passed as string to java.io.File constructor and
> File does not support the file URI, so my suggestion is trying to convert
> from string to URI at first.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17088) Failed to load Xinclude files with relative path in case of loading conf via URI
[
https://issues.apache.org/jira/browse/HADOOP-17088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151066#comment-17151066
]
Steve Loughran commented on HADOOP-17088:
-
the check for isRestricted already executes before this, so I don't see it
opening up access mori
> Failed to load Xinclude files with relative path in case of loading conf via
> URI
>
>
> Key: HADOOP-17088
> URL: https://issues.apache.org/jira/browse/HADOOP-17088
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Yushi Hayasaka
>Priority: Major
>
> When we create a configuration file, which load a external XML file with
> relative path, and try to load it via calling `Configuration.addResource`
> with `Path(URI)`, we got an error, which failed to load a external XML, after
> https://issues.apache.org/jira/browse/HADOOP-14216 is merged.
> {noformat}
> Exception in thread "main" java.lang.RuntimeException: java.io.IOException:
> Fetch fail on include for 'mountTable.xml' with no fallback while loading
> 'file:/opt/hadoop/etc/hadoop/core-site.xml'
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3021)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2973)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2848)
> at
> org.apache.hadoop.conf.Configuration.iterator(Configuration.java:2896)
> at com.company.test.Main.main(Main.java:29)
> Caused by: java.io.IOException: Fetch fail on include for 'mountTable.xml'
> with no fallback while loading 'file:/opt/hadoop/etc/hadoop/core-site.xml'
> at
> org.apache.hadoop.conf.Configuration$Parser.handleEndElement(Configuration.java:3271)
> at
> org.apache.hadoop.conf.Configuration$Parser.parseNext(Configuration.java:3331)
> at
> org.apache.hadoop.conf.Configuration$Parser.parse(Configuration.java:3114)
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3007)
> ... 4 more
> {noformat}
> The cause is that the URI is passed as string to java.io.File constructor and
> File does not support the file URI, so my suggestion is trying to convert
> from string to URI at first.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17088) Failed to load Xinclude files with relative path in case of loading conf via URI
[
https://issues.apache.org/jira/browse/HADOOP-17088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17144036#comment-17144036
]
Yushi Hayasaka commented on HADOOP-17088:
-
[~jeagles]
Thanks for the comment. But I think it is not restricted now even if the
included files are provided as an absolute path (i.e. it goes without this
patch) or the path of configuration file is provided as String, right?
> Failed to load Xinclude files with relative path in case of loading conf via
> URI
>
>
> Key: HADOOP-17088
> URL: https://issues.apache.org/jira/browse/HADOOP-17088
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Yushi Hayasaka
>Priority: Major
>
> When we create a configuration file, which load a external XML file with
> relative path, and try to load it via calling `Configuration.addResource`
> with `Path(URI)`, we got an error, which failed to load a external XML, after
> https://issues.apache.org/jira/browse/HADOOP-14216 is merged.
> {noformat}
> Exception in thread "main" java.lang.RuntimeException: java.io.IOException:
> Fetch fail on include for 'mountTable.xml' with no fallback while loading
> 'file:/opt/hadoop/etc/hadoop/core-site.xml'
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3021)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2973)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2848)
> at
> org.apache.hadoop.conf.Configuration.iterator(Configuration.java:2896)
> at com.company.test.Main.main(Main.java:29)
> Caused by: java.io.IOException: Fetch fail on include for 'mountTable.xml'
> with no fallback while loading 'file:/opt/hadoop/etc/hadoop/core-site.xml'
> at
> org.apache.hadoop.conf.Configuration$Parser.handleEndElement(Configuration.java:3271)
> at
> org.apache.hadoop.conf.Configuration$Parser.parseNext(Configuration.java:3331)
> at
> org.apache.hadoop.conf.Configuration$Parser.parse(Configuration.java:3114)
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3007)
> ... 4 more
> {noformat}
> The cause is that the URI is passed as string to java.io.File constructor and
> File does not support the file URI, so my suggestion is trying to convert
> from string to URI at first.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17088) Failed to load Xinclude files with relative path in case of loading conf via URI
[
https://issues.apache.org/jira/browse/HADOOP-17088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17143908#comment-17143908
]
Jonathan Turner Eagles commented on HADOOP-17088:
-
One important security feature is to disallow xml resources from outside of the
classpath. Does this enforce this constraint?
> Failed to load Xinclude files with relative path in case of loading conf via
> URI
>
>
> Key: HADOOP-17088
> URL: https://issues.apache.org/jira/browse/HADOOP-17088
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Yushi Hayasaka
>Priority: Major
>
> When we create a configuration file, which load a external XML file with
> relative path, and try to load it via calling `Configuration.addResource`
> with `Path(URI)`, we got an error, which failed to load a external XML, after
> https://issues.apache.org/jira/browse/HADOOP-14216 is merged.
> {noformat}
> Exception in thread "main" java.lang.RuntimeException: java.io.IOException:
> Fetch fail on include for 'mountTable.xml' with no fallback while loading
> 'file:/opt/hadoop/etc/hadoop/core-site.xml'
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3021)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2973)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2848)
> at
> org.apache.hadoop.conf.Configuration.iterator(Configuration.java:2896)
> at com.company.test.Main.main(Main.java:29)
> Caused by: java.io.IOException: Fetch fail on include for 'mountTable.xml'
> with no fallback while loading 'file:/opt/hadoop/etc/hadoop/core-site.xml'
> at
> org.apache.hadoop.conf.Configuration$Parser.handleEndElement(Configuration.java:3271)
> at
> org.apache.hadoop.conf.Configuration$Parser.parseNext(Configuration.java:3331)
> at
> org.apache.hadoop.conf.Configuration$Parser.parse(Configuration.java:3114)
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3007)
> ... 4 more
> {noformat}
> The cause is that the URI is passed as string to java.io.File constructor and
> File does not support the file URI, so my suggestion is trying to convert
> from string to URI at first.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
