Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/14445 )
Change subject: [tablet] Optimize TabletMetadata::CollectBlockIds efficiency ...................................................................... Patch Set 6: (3 comments) http://gerrit.cloudera.org:8080/#/c/14445/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/14445/4//COMMIT_MSG@14 PS4, Line 14: ectBlockIds() : oper > Linear time cost for what operations? Done http://gerrit.cloudera.org:8080/#/c/14445/4/src/kudu/tablet/tablet_metadata.cc File src/kudu/tablet/tablet_metadata.cc: http://gerrit.cloudera.org:8080/#/c/14445/4/src/kudu/tablet/tablet_metadata.cc@195 PS4, Line 195: BlockIdContainer rowset > Did you try to see what if instead of using deques a vector of pre-allocate We have to cache all rowsets_'s GetAllBlocks() return values to calculate total block id count to reserve 'block_ids', after some tests, it's worse than not reserve. http://gerrit.cloudera.org:8080/#/c/14445/4/src/kudu/tablet/tablet_metadata.cc@198 PS4, Line 198: rowset_block_ids. > I think this was one of the culprits of the bad performance with std::vecto I searched all of CollectBlockIds()'s caller, didn't find any requirement of inserting at the beginning. Now I try to modify to insert at the ending, and benchmark show that it's as good as deque, or even better! I think it's because BlockId just has an uint64_t member, it's easy and fast to allocate/reallocate a large continuous memory space to hold BlockId items. Now I will revert to use vector and just insert at the ending! -- To view, visit http://gerrit.cloudera.org:8080/14445 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7ce853e35eb7dfa9f9a099e465ea8edfaa7c4aa9 Gerrit-Change-Number: 14445 Gerrit-PatchSet: 6 Gerrit-Owner: Yingchun Lai <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yingchun Lai <[email protected]> Gerrit-Comment-Date: Fri, 15 Nov 2019 04:12:55 +0000 Gerrit-HasComments: Yes
