[
https://issues.apache.org/jira/browse/HDFS-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947622#comment-15947622
]
Chen Liang commented on HDFS-11530:
-----------------------------------
I randomly looked into some failed ones, I'm under the impression that pretty
much at least 9 out of 10 test fails were due to this:
{code}
java.lang.NullPointerException: null
at org.apache.hadoop.fs.FileUtil.fullyDeleteOnExit(FileUtil.java:118)
at org.apache.hadoop.fs.FileUtil.fullyDeleteOnExit(FileUtil.java:119)
at org.apache.hadoop.fs.FileUtil.fullyDeleteOnExit(FileUtil.java:119)
at org.apache.hadoop.fs.FileUtil.fullyDeleteOnExit(FileUtil.java:119)
at
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1980)
at
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1944)
at
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1937)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:878)
at
org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:491)
at
org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:450)
{code}
My speculation is that {{MiniDFSCluster}} also needs to be changed to use
{{DFSNetworkTopology}} accordingly, otherwise when the tests call cleanup, they
would still try to delete test directories from {{NetworkTopology}}, which is
not being used (or something else like this).
There are other test fails which does not have this though, maybe it's better
to get this one fixed somehow.
> Use HDFS specific network topology to choose datanode in
> BlockPlacementPolicyDefault
> ------------------------------------------------------------------------------------
>
> Key: HDFS-11530
> URL: https://issues.apache.org/jira/browse/HDFS-11530
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Affects Versions: 3.0.0-alpha2
> Reporter: Yiqun Lin
> Assignee: Yiqun Lin
> Attachments: HDFS-11530.001.patch, HDFS-11530.002.patch
>
>
> The work for {{chooseRandomWithStorageType}} has been merged in HDFS-11482.
> But this method is contained in new topology {{DFSNetworkTopology}} which is
> specified for HDFS. We should update this and let
> {{BlockPlacementPolicyDefault}} use the new way since the original way is
> inefficient.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
