[
https://issues.apache.org/jira/browse/HDFS-8939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14712168#comment-14712168
]
Chris Nauroth commented on HDFS-8939:
-------------------------------------
bq. ...am getting a bit concerned about the backwards compatibility of this
change...
Yes, I agree. Navigating through the twisty history, I can see that there was
concern expressed about this part of the change in the later comments of
HDFS-5321. There was a specific choice to restore the behavior of making
{{getDefaultPort}} respect the configuration as part of HDFS-6632, so I think
we need to preserve that.
FWIW, I don't think this is strictly correct for the contract of
{{getDefaultPort}}. Other implementations hard-code the return value to its
typical default instead of reading configuration. {{DistributedFileSystem}} is
hard-coded to return 8020. {{FTPFileSystem}} hard-codes 21. Sometimes we have
to let compatibility win though. I suppose that's why we're taking the
opportunity change it on trunk.
For these test failures on branch-2, shall we change {{WebHdfs}} and
{{SWebHdfs}} to call {{setConf}} on the wrapped instance? This would require
private static helper methods that do the instantiation + {{setConf}}, just to
satisfy Java's rules about the call to {{super}} being the first line in the
constructor. e.g.:
{code}
super(theUri, createWebHdfsFileSystem(conf), conf, SCHEME, false);
{code}
{code}
private static WebHdfsFileSystem createWebHdfsFileSystem(Configuration conf) {
WebHdfsFileSystem fs = new WebHdfsFileSystem();
fs.setConf(conf);
return fs;
}
{code}
> Test(S)WebHdfsFileContextMainOperations failing on branch-2
> -----------------------------------------------------------
>
> Key: HDFS-8939
> URL: https://issues.apache.org/jira/browse/HDFS-8939
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
> Affects Versions: 2.8.0
> Reporter: Jakob Homan
> Assignee: Jakob Homan
> Fix For: 2.8.0
>
> Attachments: HDFS-8939-branch-2.001.patch,
> HDFS-8939-branch-2.002.patch
>
>
> After HDFS-8180, TestWebHdfsFileContextMainOperations and
> TestSWebHdfsFileContextMainOperations are failing with runtime NPEs while
> instantiating the wrapped WebHDFSFileSystems because {{getDefaultPort}} is
> trying to access a conf that was never provided. In the constructor both
> both WebHdfs and SWebhdfs the underlying (S)WebHdfsFileSystems are
> instantiated in the constructor and never have a chance to have their
> {{setConf}} methods called:
> {code} SWebHdfs(URI theUri, Configuration conf)
> throws IOException, URISyntaxException {
> super(theUri, new SWebHdfsFileSystem(), conf, SCHEME, false);
> }r{code}
> The test passes on trunk because HDFS-5321 removed the call to the
> Configuration instance as part of {{getDefaultPort}}. HDFS-5321 was applied
> to branch-2 but reverted in HDFS-6632, so there's a bit of a difference in
> how branch-2 versus trunk handles default values (branch-2 pulls them from
> configs if specified, trunk just returns the hard-coded value from the
> constants file).
> I've fixed this behave like trunk and return just the hard-coded value, which
> causes the test to pass.
> There is no WebHdfsFileSystem that takes a Config, which would be another
> way to fix this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
