Adar Dembo has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14822 )
Change subject: log_index: use RWFiles for IO ...................................................................... log_index: use RWFiles for IO To use LogIndex in FileCache, we need to do one of two things: 1. Add an mmap-based file abstraction to Env, to be used by LogIndex. 2. Rework LogIndex to use RWFile instead of memory mappings. This patch implements option #2. Why? Because although memory mappings can be used for zero copy IO, the LogIndex wasn't doing that, and more importantly, failures during memory mapped IO are communicated via UNIX signals, making it practically impossible for an application of Kudu's complexity to recover from a WAL disk failure surfaced during log index IO, a feature that is being actively worked on in KUDU-2975. IO through mmap is identical to IO through RWFile (i.e. pwrite/pread) for all other intents and purposes: - Both can use ftruncate to grow the file's size while keeping it sparse. - Both maintain holes in file sections that aren't written. - Both go through the page cache for reads and writes. - Both allow pages to be dirty before writing them out asynchronously. Change-Id: I75c0476bbd9be55657291c85488b9121e04a91de Reviewed-on: http://gerrit.cloudera.org:8080/14822 Reviewed-by: Alexey Serbin <[email protected]> Reviewed-by: Andrew Wong <[email protected]> Tested-by: Kudu Jenkins --- M src/kudu/consensus/log-test.cc M src/kudu/consensus/log.cc M src/kudu/consensus/log_index-test.cc M src/kudu/consensus/log_index.cc M src/kudu/consensus/log_index.h 5 files changed, 72 insertions(+), 96 deletions(-) Approvals: Alexey Serbin: Looks good to me, approved Andrew Wong: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/14822 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I75c0476bbd9be55657291c85488b9121e04a91de Gerrit-Change-Number: 14822 Gerrit-PatchSet: 3 Gerrit-Owner: Adar Dembo <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Andrew Wong <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120)
