Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/11885 )
Change subject: Limit number of rowsets in compaction selection to 32 in TSAN mode ...................................................................... Patch Set 2: Todd mentioned offline that it might be a good idea to limit the number of rowsets in a compaction even in release mode, since there is a memory cost per column per rowset involved in a compaction, up to 256KB. At our column limit of 300 and the max number of rowsets in a compaction with default settings, 128, this means at worst using 300 * 128 * 256KB = 9.8GB which is a lot. With a more-typical 50 columns, it's still over a GB of memory. I'm not sure what typical usage per column would be, though. Is it a lot less than 256KB? -- To view, visit http://gerrit.cloudera.org:8080/11885 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I01ad4ba3a13995c194c3308d72c1eb9b611ef766 Gerrit-Change-Number: 11885 Gerrit-PatchSet: 2 Gerrit-Owner: Will Berkeley <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Andrew Wong <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Will Berkeley <[email protected]> Gerrit-Comment-Date: Tue, 06 Nov 2018 16:11:49 +0000 Gerrit-HasComments: No
