Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/14822 )
Change subject: log_index: use RWFiles for IO ...................................................................... Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/14822/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/14822/1//COMMIT_MSG@15 PS1, Line 15: failures during memory mapped IO > BTW, what kind of failures did we get there in Kudu? Something like not en Running out of disk space is one such example, though that's typically mitigated by --fs_wal_dir_reserved_bytes. A more common example is a disk error that causes the filesystem to be remounted in read-only mode, at which case an attempt to memcpy() into the memory mapped region manifests as a SIGBUS. http://gerrit.cloudera.org:8080/#/c/14822/1//COMMIT_MSG@20 PS1, Line 20: IO through RWFile > Do we expect less or more memory allocated in this case compared with the c In Kudu itself? A bit more per index chunk, as there's more bookkeeping in an RWFile vs. the existing memory-mapped chunk implementation. In the kernel? I'd expect the same amount. In both cases, 4k pages worth of an underlying index chunk are loaded into the page cache as they are accessed, and the page cache may dump those pages (or, if dirty, write them out) on memory pressure. http://gerrit.cloudera.org:8080/#/c/14822/1/src/kudu/consensus/log_index.cc File src/kudu/consensus/log_index.cc: http://gerrit.cloudera.org:8080/#/c/14822/1/src/kudu/consensus/log_index.cc@237 PS1, Line 237: "Unable to delete index chunk " << path; > Maybe, add information from the Status object (i.e. s.ToString()) into the Done -- To view, visit http://gerrit.cloudera.org:8080/14822 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I75c0476bbd9be55657291c85488b9121e04a91de Gerrit-Change-Number: 14822 Gerrit-PatchSet: 1 Gerrit-Owner: Adar Dembo <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Andrew Wong <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 04 Dec 2019 17:37:09 +0000 Gerrit-HasComments: Yes
