[
https://issues.apache.org/jira/browse/HDFS-6773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen Chu updated HDFS-6773:
------------------------------
Attachment: HDFS-6773.1.patch
Attaching a patch.
* Add {{skipFsyncForTesting}} builder option which defaults to true to
MiniDFSCluster.
* Remove enabling fsync in {{TestFsDatasetCache}} and {{TestCacheDirectives}}
because it's not needed.
I left the instances of
EditLogFileOutputStream.setShouldSkipFsyncForTesting(true); in this first
patch. Let me know if it's better just to remove them all, or to use the new
Builder option in some of them to let new test readers be aware of this option.
Quick scan through tests searching for "fsync" and I don't think any current
tests require fsync.
> MiniDFSCluster can run dramatically faster
> ------------------------------------------
>
> Key: HDFS-6773
> URL: https://issues.apache.org/jira/browse/HDFS-6773
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Affects Versions: 2.0.0-alpha, 3.0.0
> Reporter: Daryn Sharp
> Assignee: Stephen Chu
> Attachments: HDFS-6773.1.patch
>
>
> The mini cluster is unnecessarily running with durable edit logs. The
> following change cut runtime of a single test from ~30s to ~10s.
> {code}EditLogFileOutputStream.setShouldSkipFsyncForTesting(true);{code}
> The mini cluster should default to this behavior after identifying the few
> edit log tests that probably depend on durable logs.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
