ConfX created HADOOP-18802:
------------------------------
Summary: Collision of config key name
fs.viewfs.mounttable.default.name.key to other keys that specify the entry
point to mount tables
Key: HADOOP-18802
URL: https://issues.apache.org/jira/browse/HADOOP-18802
Project: Hadoop Common
Issue Type: Bug
Components: common, conf, fs
Reporter: ConfX
Attachments: reproduce.sh
h2. What happened:
When manually set {{fs.viewfs.mounttable.default.name.key}} to {{default}} (the
same as default value) in HCommon, test
{{org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs#testGlobStatusWithMultipleWildCardMatches}}
would fail.
But the test can pass if this parameter is not manually set in the
configuration file.
h2. Where's the bug:
In the constructor of {{{}InodeTree{}}}, the tree attempts to get all the mount
table entry points set by user in the configuration and process them one by one:
for (Entry<String, String> si : config) { final String key =
si.getKey(); if (!key.startsWith(mountTablePrefix)) { continue;
} gotMountTableEntry = true; LinkType linkType; String src
= key.substring(mountTablePrefix.length());
...
Here {{{}mountTablePrefix="fs.viewfs.mounttable.default."{}}}. However, it just
so happens that the name of the configuration users use to specify the default
mount table is {{{}fs.viewfs.mounttable.default.name.key{}}}. Thus, if a user
specifies the default mount table and uses InodeTree the {{name.key}} would be
falsely parsed as the entry point to one of the mount tables, which would cause
{{InodeTree}} to throw an exception since {{name.key}} is not a valid entry.
h2. Stack trace:
java.lang.RuntimeException: java.io.IOException: ViewFs: Cannot initialize:
Invalid entry in Mount table in config: name.keyat
org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) at
org.apache.hadoop.fs.viewfs.ViewFsTestSetup.setupForViewFsLocalFs(ViewFsTestSetup.java:88)
at
org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs.setUp(TestFcMainOperationsLocalFs.java:38)
h2. How to reproduce:
(1) Set {{fs.viewfs.mounttable.default.name.key}} to {{default}}
(2) Run test
{{org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs#testGlobStatusWithMultipleWildCardMatches}}
You can use the reproduce.sh in the attachment to easily reproduce the bug:
We are happy to provide a patch if this issue is confirmed.
{{}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
