[
https://issues.apache.org/jira/browse/HDFS-16910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shilun Fan updated HDFS-16910:
------------------------------
Component/s: namanode
> Fix incorrectly initializing RandomAccessFile caused flush performance
> decreased for JN
> ---------------------------------------------------------------------------------------
>
> Key: HDFS-16910
> URL: https://issues.apache.org/jira/browse/HDFS-16910
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namanode
> Affects Versions: 3.4.0
> Reporter: Haiyang Hu
> Assignee: Haiyang Hu
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> At present, after our cluster backport patch HDFS-15882,
> when set shouldSyncWritesAndSkipFsync to false, there will be flush
> performance degradation caused by JN.
> *Root Cause*：
> when setting shouldSyncWritesAndSkipFsync to false, the mode of init
> RandomAccessFile will be `rws`.
> even if fc.force(false) is executed when flushAndSync is executed (hopefully,
> only requires updates to the file's content to be written to storage and the
> metadata is not update),
> but since the mode of RandomAccessFile is `rws`, It will requires updates to
> both the file's content and its metadata to be written,
> there will be flush performance degradation caused by JN.
> *Fix:*
> Need to update RandomAccessFile's mode from `rws` to `rwd`:
> rwd: Open for reading and writing, as with "rw", and also require that every
> update to the file's content be written synchronously to the underlying
> storage device.
> {code:java}
> if (shouldSyncWritesAndSkipFsync) {
> rp = new RandomAccessFile(name, "rwd");
> } else {
> rp = new RandomAccessFile(name, "rw");
> }
> {code}
> In this way, when flushAndSync is executed,
> if shouldSyncWritesAndSkipFsync is false and the mode of RandomAccessFile is
> 'rw', it will call fc.force(false) to execute,
> otherwise should use `rwd` to perform the operation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
