[
https://issues.apache.org/jira/browse/HDFS-17794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18041328#comment-18041328
]
ASF GitHub Bot commented on HDFS-17794:
---------------------------------------
dlmarion commented on code in PR #8065:
URL: https://github.com/apache/hadoop/pull/8065#discussion_r2571985726
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml:
##########
@@ -1779,6 +1779,21 @@
</description>
</property>
+<property>
+ <name>dfs.datanode.synconclose</name>
+ <value>false</value>
+ <description>
+ If this configuration is enabled, the datanode will instruct the
+ operating system to sync data block files contents to disk when the
+ blockfile is closed.
+
+ This has the benefit of ensuring that data is written to disk
+ immediately when a block is closed, avoiding the data being held
+ in the operating system cache. This can help data loss in the event
+ of a malfunction such as a power failure.
Review Comment:
> ..., avoiding the data being held in the operating system cache.
Does the sync of the dirty block in the page cache force the removal of the
block from the page cache? I'm not sure that it does. The user can advise the
OS, where supported, that it doesn't need the block in the page cache through
the `dfs.datanode.drop.cache.behind.writes` property or related flags when
opening the file.
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml:
##########
@@ -1779,6 +1779,21 @@
</description>
</property>
+<property>
+ <name>dfs.datanode.synconclose</name>
+ <value>false</value>
+ <description>
+ If this configuration is enabled, the datanode will instruct the
Review Comment:
```suggestion
If this property is enabled, the datanode will instruct the
```
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml:
##########
@@ -1779,6 +1779,21 @@
</description>
</property>
+<property>
+ <name>dfs.datanode.synconclose</name>
+ <value>false</value>
+ <description>
+ If this configuration is enabled, the datanode will instruct the
+ operating system to sync data block files contents to disk when the
+ blockfile is closed.
Review Comment:
```suggestion
blockfile is closed. This can help data loss in the event of a
malfunction such as a power
failure.
```
> Document the property dfs.datanode.synconclose
> ----------------------------------------------
>
> Key: HDFS-17794
> URL: https://issues.apache.org/jira/browse/HDFS-17794
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: documentation
> Affects Versions: 3.4.1
> Reporter: Jim Halfpenny
> Assignee: Jim Halfpenny
> Priority: Minor
> Labels: pull-request-available
>
> The hdfs-site.xml property dfs.datanode.synconclose appears in the HDFS code
> but is not documented. The property, it's default value and the behaviour of
> this configuration property should be added to the Hadoop HDFS documentation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
