[
https://issues.apache.org/jira/browse/HDFS-10610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371669#comment-15371669
]
Daryn Sharp commented on HDFS-10610:
------------------------------------
Although not strictly HDFS-8314's fault, it's the other half of the "2 wrongs
accidentally make a right".
Simplest answer is remove {{@InterfaceAudience.Private}} from
{{HdfsConfiguration}} otherwise there's no clean way to trigger loading
hdfs-site w/o explicitly adding the resource.
Spark complained about the visibility of this class when they encountered a
similar problem. In their case they pass serialized confs. If a filesystem
isn't accessed prior to serializing the conf - because the fs service loader
will trigger creation of
{{DistributedFileSystem}}/{{DFSClient}}/{{HdfsConfiguration}} - hdfs parameters
are missing. Very confusing to debug.
> DfsClient doesn't add hdfs-site.xml as a resource
> -------------------------------------------------
>
> Key: HDFS-10610
> URL: https://issues.apache.org/jira/browse/HDFS-10610
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 2.8.0
> Reporter: Eric Badger
>
> Instantiating a new DfsClient used to add hdfs-site.xml as a resource in 2.7,
> but that compatibility has been broken in 2.8. This only accidentally worked
> in 2.7, since DfsClient would load HdfsConstants, which would in turn create
> the static IO_FILE_BUFFER_SIZE, which would instantiate an HdfsConfiguration,
> which would finally add hdfs-site.xml.
> In 2.8, IO_FILE_BUFFER_SIZE no longer exists in HdfsConstants.java and so
> this no longer works out of coincidence.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
