Todd Lipcon has submitted this change and it was merged. Change subject: cfile_set: reduce memory usage of reader map ......................................................................
cfile_set: reduce memory usage of reader map We previously used an unordered_map<int, shared_ptr<...>>. This patch changes to using boost's flat_map container, which is based on sorted entries in a vector. This means we get one contiguous memory location vs a bunch of smaller per-item allocations in the separate-chained hashtable. Additionally switches to unique_ptr which avoids an extra 16-byte control structure per CFileReader. It's hard to measure the exact win here, but I wrote a little test program which compared inserting 40 entries into the two types of maps, leaking the maps, and looking at the LeakSanitizer reports. With the old map and shared_ptr, it cost 2832 bytes, and with the flat_map and unique_ptr, it only took 824 bytes. So, we saved about 50 bytes per entry. Extrapolating this out to a server with 600k cfiles open, we'll save around 28MB. Not huge, but worth doing. Change-Id: I3812861a11b00e149ca47b090447bc918c465ad4 Reviewed-on: http://gerrit.cloudera.org:8080/6466 Reviewed-by: Adar Dembo <a...@cloudera.com> Tested-by: Kudu Jenkins --- M src/kudu/tablet/cfile_set.cc M src/kudu/tablet/cfile_set.h 2 files changed, 9 insertions(+), 6 deletions(-) Approvals: Adar Dembo: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/6466 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I3812861a11b00e149ca47b090447bc918c465ad4 Gerrit-PatchSet: 4 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Todd Lipcon <t...@apache.org> Gerrit-Reviewer: Adar Dembo <a...@cloudera.com> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon <t...@apache.org>
