Ivan Andika created HDDS-14246:
----------------------------------
Summary: Optionally sync write during PutBlock
Key: HDDS-14246
URL: https://issues.apache.org/jira/browse/HDDS-14246
Project: Apache Ozone
Issue Type: Sub-task
Reporter: Ivan Andika
Assignee: Ivan Andika
Currently, datanode has an option to flush the write on chunk boundary
(hdds.container.chunk.write.sync) which is disabled by default since it might
affect the DN write throughput and latency. However, disabling this means that
if the datanode machine is suddenly down (e.g. power failure, reaped by OOM
killer), this might cause the file to have incomplete data even if PutBlock
(write commit) is successful. Although PutBlock triggers
FilePerBlockStrategy#finishWriteChunks which will trigger close
RandomAccessFile#close, the buffer cache might not be flushed yet (see
https://man7.org/linux/man-pages/man2/close.2.html). This means that there
might be mismatch between the block metadata and the actual data which can
cause data inconsistency and loss.
We might need to consider calling FileChannel#force on PutBlock instead of
WriteChunk. This might still affect the write throughput and latency, but will
increase our data durability guarantee.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
