Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/19246 )
Change subject: WIP [compact] Increase chances of compaction for large number deltas ...................................................................... Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/19246/1/src/kudu/tablet/delta_tracker.cc File src/kudu/tablet/delta_tracker.cc: http://gerrit.cloudera.org:8080/#/c/19246/1/src/kudu/tablet/delta_tracker.cc@513 PS1, Line 513: undo_redo->EstimateSize(); > I guess you are talking about data at rest is in compressed format and can IIRC, cfile might contain compressed blocks, and from the memory profiles attached to KUDU-3406 you can see CFileReader::ReadBlock() is called from DeltaFileIterator<Type>::ReadCurrentBlockOntoQueue(). CFileReader::ReadBlock() performs uncompression, and allocates the memory for the uncompressed data prior to that. So, I was curious: whether the Estimate() for deltas return the size uncompressed (as represented in memory) or compressed (as represented on disk). Everything will be much simpler if the estimate is for the former, not the latter. If that's the latter, some adjustment is needed. http://gerrit.cloudera.org:8080/#/c/19246/1/src/kudu/tablet/mock-rowsets.h File src/kudu/tablet/mock-rowsets.h: http://gerrit.cloudera.org:8080/#/c/19246/1/src/kudu/tablet/mock-rowsets.h@162 PS1, Line 162: EstimateBytesInDeltas > Are you pointing to TestYcsbCompaction test? I meant tests in compaction_policy-test.cc, but those might be rather testing outcomes of so-called budgeted compaction policy. http://gerrit.cloudera.org:8080/#/c/19246/1/src/kudu/tablet/tablet.cc File src/kudu/tablet/tablet.cc: http://gerrit.cloudera.org:8080/#/c/19246/1/src/kudu/tablet/tablet.cc@2469 PS1, Line 2469: double workload_score = 0; > If you are talking about workload_score, I think it may make more sense to That was about max_limit_deltas_size, of course. -- To view, visit http://gerrit.cloudera.org:8080/19246 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2199ae3b777e75b15b60d8ad818cc6adc4f5fa3b Gerrit-Change-Number: 19246 Gerrit-PatchSet: 1 Gerrit-Owner: Ashwani Raina <[email protected]> Gerrit-Reviewer: Abhishek Chennaka <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Ashwani Raina <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 16 Nov 2022 06:07:08 +0000 Gerrit-HasComments: Yes
