David Ribeiro Alves has submitted this change and it was merged. Change subject: Don't output unobservable rows from the MemRowset ......................................................................
Don't output unobservable rows from the MemRowset In some rare cases we might have a row that is inserted and deleted in the same operation, meaning the insert and delete have the same timestamp. In this rare case we might run into trouble later on with compactions as there is a sequence of operations that that produce two ghost rows which are uncomparable in terms of recency. The sequence is as follows: Insert Row at 1 -> Row goes in the MRS Flush -> Row now lives in RS1 Delete Row at 3 -> Row is now a ghost in RS1 Insert Row at 3 -> Row goes in the MRS again Delete Row at 3 -> Row is deleted from the MRS Flush -> Row is now a ghost in RS2 Major delta compact RS1 -> GC all before 3 Major delta compact RS2 -> GC all before 3 If RS1 and RS2 now get compacted together there is no way to distinguish Ghost 1 from Ghost 2 and to build a correct history for the row. The point is that we can't trust UNDOs to break ties, in terms of recency, between two ghosts as they might have been removed by delta compactions. However we can always trust REDO delete/reinsert to remain. This adds a small test to compaction-test that makes sure that a row that is inserted and deleted in the same transaction doesn't appear in the compaction input. FuzzTest and TestRandomAccess would fail for the following patch (KUDU-237 (part 2)) without this change. Change-Id: Ieab52a1aa68494218f91f3acd31ef8ddf352bd57 Reviewed-on: http://gerrit.cloudera.org:8080/4994 Tested-by: Kudu Jenkins Reviewed-by: David Ribeiro Alves <[email protected]> Reviewed-by: Mike Percy <[email protected]> --- M src/kudu/tablet/compaction-test.cc M src/kudu/tablet/compaction.cc 2 files changed, 185 insertions(+), 62 deletions(-) Approvals: David Ribeiro Alves: Looks good to me, but someone else must approve Mike Percy: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/4994 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ieab52a1aa68494218f91f3acd31ef8ddf352bd57 Gerrit-PatchSet: 18 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: David Ribeiro Alves <[email protected]> Gerrit-Reviewer: David Ribeiro Alves <[email protected]> Gerrit-Reviewer: Jean-Daniel Cryans <[email protected]> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy <[email protected]> Gerrit-Reviewer: Tidy Bot Gerrit-Reviewer: Todd Lipcon <[email protected]>
