[
https://issues.apache.org/jira/browse/HADOOP-18802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ConfX updated HADOOP-18802:
---------------------------
Description:
h2. What happened:
When manually set fs.viewfs.mounttable.default.name.key to default (the same as
default value) in HCommon, test
org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs#testGlobStatusWithMultipleWildCardMatches
would fail.
But the test can pass if this parameter is not manually set in the
configuration file.
h2. Where's the bug:
In the constructor of {{{}InodeTree{}}}, the tree attempts to get all the mount
table entry points set by user in the configuration and process them one by one:
{code:java}
for (Entry<String, String> si : config) {
final String key = si.getKey();
if (!key.startsWith(mountTablePrefix)) {
continue;
}
gotMountTableEntry = true;
LinkType linkType;
String src = key.substring(mountTablePrefix.length());
...
{code}
Here {{{}mountTablePrefix="fs.viewfs.mounttable.default."{}}}. However, it just
so happens that the name of the configuration users use to specify the default
mount table is {{{}fs.viewfs.mounttable.default.name.key{}}}. Thus, if a user
specifies the default mount table and uses InodeTree the {{name.key}} would be
falsely parsed as the entry point to one of the mount tables, which would cause
{{InodeTree}} to throw an exception since {{name.key}} is not a valid entry.
h2. Stack trace:
{code:java}
java.lang.RuntimeException: java.io.IOException: ViewFs: Cannot initialize:
Invalid entry in Mount table in config: name.key
at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
at
org.apache.hadoop.fs.viewfs.ViewFsTestSetup.setupForViewFsLocalFs(ViewFsTestSetup.java:88)
at
org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs.setUp(TestFcMainOperationsLocalFs.java:38){code}
h2. How to reproduce:
(1) Set {{fs.viewfs.mounttable.default.name.key}} to {{default}}
(2) Run test
{{org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs#testGlobStatusWithMultipleWildCardMatches}}
You can use the reproduce.sh in the attachment to easily reproduce the bug:
We are happy to provide a patch if this issue is confirmed.
was:
h2. What happened:
When manually set {{fs.viewfs.mounttable.default.name.key}} to {{default}} (the
same as default value) in HCommon, test
{{org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs#testGlobStatusWithMultipleWildCardMatches}}
would fail.
But the test can pass if this parameter is not manually set in the
configuration file.
h2. Where's the bug:
In the constructor of {{{}InodeTree{}}}, the tree attempts to get all the mount
table entry points set by user in the configuration and process them one by one:
{code:java}
for (Entry<String, String> si : config) {
final String key = si.getKey();
if (!key.startsWith(mountTablePrefix)) {
continue;
}
gotMountTableEntry = true;
LinkType linkType;
String src = key.substring(mountTablePrefix.length());
...
{code}
Here {{{}mountTablePrefix="fs.viewfs.mounttable.default."{}}}. However, it just
so happens that the name of the configuration users use to specify the default
mount table is {{{}fs.viewfs.mounttable.default.name.key{}}}. Thus, if a user
specifies the default mount table and uses InodeTree the {{name.key}} would be
falsely parsed as the entry point to one of the mount tables, which would cause
{{InodeTree}} to throw an exception since {{name.key}} is not a valid entry.
h2. Stack trace:
{code:java}
java.lang.RuntimeException: java.io.IOException: ViewFs: Cannot initialize:
Invalid entry in Mount table in config: name.key
at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
at
org.apache.hadoop.fs.viewfs.ViewFsTestSetup.setupForViewFsLocalFs(ViewFsTestSetup.java:88)
at
org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs.setUp(TestFcMainOperationsLocalFs.java:38){code}
h2. How to reproduce:
(1) Set {{fs.viewfs.mounttable.default.name.key}} to {{default}}
(2) Run test
{{org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs#testGlobStatusWithMultipleWildCardMatches}}
You can use the reproduce.sh in the attachment to easily reproduce the bug:
We are happy to provide a patch if this issue is confirmed.
{{}}
> Collision of config key name fs.viewfs.mounttable.default.name.key to other
> keys that specify the entry point to mount tables
> -----------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-18802
> URL: https://issues.apache.org/jira/browse/HADOOP-18802
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, conf, fs
> Reporter: ConfX
> Priority: Critical
> Attachments: reproduce.sh
>
>
> h2. What happened:
> When manually set fs.viewfs.mounttable.default.name.key to default (the same
> as default value) in HCommon, test
> org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs#testGlobStatusWithMultipleWildCardMatches
> would fail.
> But the test can pass if this parameter is not manually set in the
> configuration file.
> h2. Where's the bug:
> In the constructor of {{{}InodeTree{}}}, the tree attempts to get all the
> mount table entry points set by user in the configuration and process them
> one by one:
> {code:java}
> for (Entry<String, String> si : config) {
> final String key = si.getKey();
> if (!key.startsWith(mountTablePrefix)) {
> continue;
> }
>
> gotMountTableEntry = true;
> LinkType linkType;
> String src = key.substring(mountTablePrefix.length());
> ...
> {code}
> Here {{{}mountTablePrefix="fs.viewfs.mounttable.default."{}}}. However, it
> just so happens that the name of the configuration users use to specify the
> default mount table is {{{}fs.viewfs.mounttable.default.name.key{}}}. Thus,
> if a user specifies the default mount table and uses InodeTree the
> {{name.key}} would be falsely parsed as the entry point to one of the mount
> tables, which would cause {{InodeTree}} to throw an exception since
> {{name.key}} is not a valid entry.
> h2. Stack trace:
> {code:java}
> java.lang.RuntimeException: java.io.IOException: ViewFs: Cannot initialize:
> Invalid entry in Mount table in config: name.key
> at
> org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
> at
> org.apache.hadoop.fs.viewfs.ViewFsTestSetup.setupForViewFsLocalFs(ViewFsTestSetup.java:88)
> at
> org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs.setUp(TestFcMainOperationsLocalFs.java:38){code}
> h2. How to reproduce:
> (1) Set {{fs.viewfs.mounttable.default.name.key}} to {{default}}
> (2) Run test
> {{org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs#testGlobStatusWithMultipleWildCardMatches}}
> You can use the reproduce.sh in the attachment to easily reproduce the bug:
> We are happy to provide a patch if this issue is confirmed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
