[
https://issues.apache.org/jira/browse/HDDS-1843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Banerjee updated HDDS-1843:
--------------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Undetectable corruption after restart of a datanode
> ---------------------------------------------------
>
> Key: HDDS-1843
> URL: https://issues.apache.org/jira/browse/HDDS-1843
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone Datanode
> Affects Versions: 0.5.0
> Reporter: Shashikant Banerjee
> Assignee: Shashikant Banerjee
> Priority: Critical
> Labels: pull-request-available
> Fix For: 0.5.0
>
> Attachments: HDDS-1843.000.patch
>
> Time Spent: 9h 50m
> Remaining Estimate: 0h
>
> Right now, all write chunks use BufferedIO ie, sync flag is disabled by
> default. Also, Rocks Db metadata updates are done in Rocks DB cache first at
> Datanode. In case, there comes a situation where the buffered chunk data as
> well as the corresponding metadata update is lost as a part of datanode
> restart, it may lead to a situation where, it will not be possible to detect
> the corruption (not even with container scanner) of this nature in a
> reasonable time frame, until and unless there is a client IO failure or Recon
> server detects it over time. In order to atleast to detect the problem, Ratis
> snapshot on datanode should sync the rocks db file . In such a way,
> ContainerScanner will be able to detect this.We can also add a metric around
> sync to measure how much of a throughput loss it can incurr.
> Thanks [~msingh] for suggesting this.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
