David Ribeiro Alves has submitted this change and it was merged. Change subject: KUDU-237 (part 2) - Add support for REINSERT in delta files ......................................................................
KUDU-237 (part 2) - Add support for REINSERT in delta files This patch goes the final mile in adding support for REINSERTs in delta files. It does various things to achieve this: - Transforms REDO DELETE/REINSERT mutation pairs into UNDO REINSERT/DELETE mutation pairs, leaving at the most a REDO delete. - Merges ghost rows on compaction: When duplicated rows are found a new algorithm finds out which one is the most recent and adds the other one as a 'previous_ghost'. This can happen for an arbitrary number of ghost rows. Noteworthy is that setting previous versions requires a copy. The two rows are in different RowBlocks (for row data) and Arenas (for mutations) and it is not guaranteed that the previous ghost will suvive by the time the row that points to it is processed. - Adds new test to tablet-test and changes a test in compaction-test proving that this works. - Adds a new test to compaction-test that creates several layers of overlapping rowsets where each layer has one rowset less than the previous one. This creates a mix of duplicated (up to 10 different ghosts) and unique rows. The test then compacts the rowsets a few at a time and makes sure the histories are correct. Follow up patches will add new itests that test this functionality even more broadly. Ran 500 loops of mt-tablet-test and compaction-test in dist-test with "KUDU_ALLOW_SLOW_TESTS=1" and "--stress_cpu_threads=4". All tests passed. Results: compaction-test: http://dist-test.cloudera.org//job?job_id=david.alves.1480028902.31041 mt-tablet-test: http://dist-test.cloudera.org//job?job_id=david.alves.1480028908.31125 Change-Id: Ie1173b2bea721b376f2b6049be20f57307582c47 Reviewed-on: http://gerrit.cloudera.org:8080/4995 Tested-by: Kudu Jenkins Reviewed-by: David Ribeiro Alves <[email protected]> Reviewed-by: Mike Percy <[email protected]> --- M src/kudu/common/row_changelist.h M src/kudu/tablet/compaction-test.cc M src/kudu/tablet/compaction.cc M src/kudu/tablet/compaction.h M src/kudu/tablet/delta_store.cc M src/kudu/tablet/deltafile.cc M src/kudu/tablet/deltafile.h M src/kudu/tablet/tablet-test.cc 8 files changed, 634 insertions(+), 134 deletions(-) Approvals: David Ribeiro Alves: Looks good to me, approved Mike Percy: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/4995 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ie1173b2bea721b376f2b6049be20f57307582c47 Gerrit-PatchSet: 25 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: David Ribeiro Alves <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: David Ribeiro Alves <[email protected]> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy <[email protected]> Gerrit-Reviewer: Tidy Bot Gerrit-Reviewer: Todd Lipcon <[email protected]>
