[
https://issues.apache.org/jira/browse/HDFS-16910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17685700#comment-17685700
]
ASF GitHub Bot commented on HDFS-16910:
---------------------------------------
ZanderXu commented on code in PR #5359:
URL: https://github.com/apache/hadoop/pull/5359#discussion_r1099712434
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileOutputStream.java:
##########
@@ -84,9 +84,9 @@ public EditLogFileOutputStream(Configuration conf, File name,
int size)
doubleBuf = new EditsDoubleBuffer(size);
RandomAccessFile rp;
if (shouldSyncWritesAndSkipFsync) {
Review Comment:
`shouldSyncWritesAndSkipFsync=true` means that we will use
`RandomAccessFile(name, "rwd")` to sync data instead of `fc.force(false)`.
`shouldSyncWritesAndSkipFsync=false` means that we will use
`fc.force(false)` to sync data, here `rp = new RandomAccessFile(name, "rw");`
> Fix incorrectly initializing RandomAccessFile caused flush performance
> decreased for JN
> ---------------------------------------------------------------------------------------
>
> Key: HDFS-16910
> URL: https://issues.apache.org/jira/browse/HDFS-16910
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Haiyang Hu
> Assignee: Haiyang Hu
> Priority: Major
> Labels: pull-request-available
>
> At present, after our cluster backport patch HDFS-15882,
> when set shouldSyncWritesAndSkipFsync to false, there will be flush
> performance degradation caused by JN.
> *Root Cause*：
> when setting shouldSyncWritesAndSkipFsync to false, the mode of init
> RandomAccessFile will be `rws`.
> even if fc.force(false) is executed when flushAndSync is executed (hopefully,
> only requires updates to the file's content to be written to storage and the
> metadata is not update),
> but since the mode of RandomAccessFile is `rws`, It will requires updates to
> both the file's content and its metadata to be written,
> there will be flush performance degradation caused by JN.
> *Fix:*
> Need to update RandomAccessFile's mode from `rws` to `rwd`:
> rwd: Open for reading and writing, as with "rw", and also require that every
> update to the file's content be written synchronously to the underlying
> storage device.
> {code:java}
> if (shouldSyncWritesAndSkipFsync) {
> rp = new RandomAccessFile(name, "rwd");
> } else {
> rp = new RandomAccessFile(name, "rw");
> }
> {code}
> In this way, when flushAndSync is executed,
> if shouldSyncWritesAndSkipFsync is false and the mode of RandomAccessFile is
> 'rw', it will call fc.force(false) to execute,
> otherwise should use `rwd` to perform the operation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
