[
https://issues.apache.org/jira/browse/HDFS-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16735717#comment-16735717
]
Behroz Sikander commented on HDFS-6360:
---------------------------------------
Are there any plans to fix this issue?
We are also facing unexpected behavior of testcases.
> MiniDFSCluster can cause unexpected side effects due to sharing of config
> -------------------------------------------------------------------------
>
> Key: HDFS-6360
> URL: https://issues.apache.org/jira/browse/HDFS-6360
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Kihwal Lee
> Priority: Major
>
> As noted in HDFS-6329 and HDFS-5522, certain use cases of MiniDFSCluster can
> result in unexpected results and falsely failing or passing unit tests.
> Since a {{Configuration}} object is shared for all namenode startups, the
> modified conf object during a NN startup is passed to the next NN startup.
> The effect of the modified conf propagation and subsequent modifications is
> different depending on whether it is a single NN cluster, HA cluster or
> federation cluster.
> It also depends on what test cases are doing with the config. For example,
> MiniDFSCluster#getConfiguration(int) returns the saved conf for the specified
> NN, but that is not actually the conf object used by the NN. It just
> contained the same content one time in the past and it is not guaranteed to
> be that way.
> Restarting the same NN can also cause unexpected results. The new NN will
> switch to the conf that was cloned & saved AFTER the last startup. The new
> NN will start with a changed config intentionally or unintentionally. The
> config variables such as {{fs.defaultFs}}, {{dfs.namenode.rpc-address}} will
> be implicitly set differently than the initial condition. Some test cases
> rely on this and others occasionally break because of this.
> In summary,
> * MiniDFSCluster does not properly isolate configs.
> * Many test cases happen to work most of times. Correcting MiniDFSCluster
> causes mass breakages of test cases and requires fixing them.
> * Many test cases rely on broken behavior and might pass when they should
> have actually failed.
> We need to
> * Make MiniDFSCluster behave in a consistent way
> * Provide proper methods and documentation for the correct usage of
> MiniDFSCluster
> * Fix the unit tests that will be broken after improving MiniDFSCluster.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
