Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/18503 )
Change subject: [compaction] Fix delta file with full of delete op can not be schedule to compact ...................................................................... Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/18503/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/18503/2//COMMIT_MSG@9 PS2, Line 9: If we get a REDO delta with full of delete op, wich means there is no update op : in the file. The current compact algorithm will not schedule the file do compact. : If such files exist, after accumulating for a period of time, it will greatly : affect our scan speed. However, processing such files every time compact reduces : compact's performance. Could you please create a Kudu JIRA to track this issue, describe what you see, reviewers could get more info, and some other Kudu users may happend to see this JIRA and know how to resolve it. http://gerrit.cloudera.org:8080/#/c/18503/2//COMMIT_MSG@15 PS2, Line 15: So I added a flag FLAGS_ignore_delete_op_for_compact to solve this situation. : When we found a similar situation, we can schedule the compact algorithm to deal : with REDO deltas by set FLAGS_ignore_delete_op_for_compact to false. When we solve : this problem, we need to restore the flag to true to ensure that the compact : performance will not be affected. Add a flag to control which logic to execute maybe a bit of complex: Kudu administrators should first troubleshooting this issue and then change the flag, after this issue been dealt, restore this flag. Is it possible to remove this flag, and do this work automatically? -- To view, visit http://gerrit.cloudera.org:8080/18503 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b26737dffecc17688b42188da959b2ba16351ed Gerrit-Change-Number: 18503 Gerrit-PatchSet: 2 Gerrit-Owner: KeDeng <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yingchun Lai <[email protected]> Gerrit-Comment-Date: Mon, 09 May 2022 04:31:18 +0000 Gerrit-HasComments: Yes
