[kudu-CR] log block manager: use move semantics to fill in the block map
Todd Lipcon has submitted this change and it was merged. Change subject: log_block_manager: use move semantics to fill in the block map .. log_block_manager: use move semantics to fill in the block map This avoids an extra ref-count increment/decrement while loading the block map during startup. This substantially improves startup time. This shaved off another few percent from LBM startup time. On a real host with 11M blocks: Before: I0907 17:20:42.277474 10021 fs_manager.cc:335] Time spent opening block manager: real 14.348s user 0.000s sys 0.001s After: I0907 17:27:13.600186 14550 fs_manager.cc:335] Time spent opening block manager: real 13.401s user 0.000s sys 0.002s According to the benchmark: Before: I0907 17:15:50.645310 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.570s user 0.034s sys 0.001s I0907 17:15:52.195543 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.550s user 0.037s sys 0.001s I0907 17:15:53.755209 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.560s user 0.037s sys 0.001s I0907 17:15:55.263762 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.509s user 0.038s sys 0.001s I0907 17:15:56.818748 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.555s user 0.037s sys 0.001s I0907 17:15:58.379680 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.561s user 0.036s sys 0.001s I0907 17:15:59.913751 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.534s user 0.038s sys 0.000s I0907 17:16:01.461668 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.548s user 0.037s sys 0.001s I0907 17:16:03.020823 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.559s user 0.037s sys 0.001s I0907 17:16:04.549747 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.529s user 0.035s sys 0.001s After: I0907 17:07:40.230087 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.495s user 0.032s sys 0.001s I0907 17:07:41.642105 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.412s user 0.044s sys 0.001s I0907 17:07:43.082852 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.441s user 0.040s sys 0.001s I0907 17:07:44.512603 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.430s user 0.043s sys 0.000s I0907 17:07:45.956848 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.444s user 0.041s sys 0.000s I0907 17:07:47.386735 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.430s user 0.037s sys 0.002s I0907 17:07:48.787317 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.401s user 0.041s sys 0.002s I0907 17:07:50.244407 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.457s user 0.038s sys 0.001s I0907 17:07:51.682209 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.438s user 0.037s sys 0.001s I0907 17:07:53.116209 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.434s user 0.037s sys 0.000s Change-Id: I8fe65087585e118f91320ec39f537b5f3ad6552f Reviewed-on: http://gerrit.cloudera.org:8080/8008 Reviewed-by: Adar Dembo Tested-by: Kudu Jenkins --- M src/kudu/fs/log_block_manager.cc M src/kudu/fs/log_block_manager.h 2 files changed, 14 insertions(+), 6 deletions(-) Approvals: Adar Dembo: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/8008 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I8fe65087585e118f91320ec39f537b5f3ad6552f Gerrit-PatchSet: 3 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Dan Burkert Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon
[kudu-CR] log block manager: use move semantics to fill in the block map
Adar Dembo has posted comments on this change. Change subject: log_block_manager: use move semantics to fill in the block map .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/8008 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I8fe65087585e118f91320ec39f537b5f3ad6552f Gerrit-PatchSet: 2 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Dan Burkert Gerrit-Reviewer: Kudu Jenkins Gerrit-HasComments: No
[kudu-CR] log block manager: use move semantics to fill in the block map
Hello Adar Dembo, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/8008 to look at the new patch set (#2). Change subject: log_block_manager: use move semantics to fill in the block map .. log_block_manager: use move semantics to fill in the block map This avoids an extra ref-count increment/decrement while loading the block map during startup. This substantially improves startup time. This shaved off another few percent from LBM startup time. On a real host with 11M blocks: Before: I0907 17:20:42.277474 10021 fs_manager.cc:335] Time spent opening block manager: real 14.348s user 0.000s sys 0.001s After: I0907 17:27:13.600186 14550 fs_manager.cc:335] Time spent opening block manager: real 13.401s user 0.000s sys 0.002s According to the benchmark: Before: I0907 17:15:50.645310 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.570s user 0.034s sys 0.001s I0907 17:15:52.195543 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.550s user 0.037s sys 0.001s I0907 17:15:53.755209 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.560s user 0.037s sys 0.001s I0907 17:15:55.263762 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.509s user 0.038s sys 0.001s I0907 17:15:56.818748 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.555s user 0.037s sys 0.001s I0907 17:15:58.379680 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.561s user 0.036s sys 0.001s I0907 17:15:59.913751 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.534s user 0.038s sys 0.000s I0907 17:16:01.461668 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.548s user 0.037s sys 0.001s I0907 17:16:03.020823 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.559s user 0.037s sys 0.001s I0907 17:16:04.549747 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.529s user 0.035s sys 0.001s After: I0907 17:07:40.230087 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.495s user 0.032s sys 0.001s I0907 17:07:41.642105 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.412s user 0.044s sys 0.001s I0907 17:07:43.082852 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.441s user 0.040s sys 0.001s I0907 17:07:44.512603 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.430s user 0.043s sys 0.000s I0907 17:07:45.956848 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.444s user 0.041s sys 0.000s I0907 17:07:47.386735 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.430s user 0.037s sys 0.002s I0907 17:07:48.787317 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.401s user 0.041s sys 0.002s I0907 17:07:50.244407 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.457s user 0.038s sys 0.001s I0907 17:07:51.682209 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.438s user 0.037s sys 0.001s I0907 17:07:53.116209 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.434s user 0.037s sys 0.000s Change-Id: I8fe65087585e118f91320ec39f537b5f3ad6552f --- M src/kudu/fs/log_block_manager.cc M src/kudu/fs/log_block_manager.h 2 files changed, 14 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/08/8008/2 -- To view, visit http://gerrit.cloudera.org:8080/8008 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8fe65087585e118f91320ec39f537b5f3ad6552f Gerrit-PatchSet: 2 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Dan Burkert Gerrit-Reviewer: Kudu Jenkins
[kudu-CR] log block manager: use move semantics to fill in the block map
Adar Dembo has posted comments on this change. Change subject: log_block_manager: use move semantics to fill in the block map .. Patch Set 1: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/8008/1/src/kudu/fs/log_block_manager.cc File src/kudu/fs/log_block_manager.cc: PS1, Line 2210: UntrackedBlockMap::value_type Nit: change to auto, if you're interested. -- To view, visit http://gerrit.cloudera.org:8080/8008 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I8fe65087585e118f91320ec39f537b5f3ad6552f Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Dan Burkert Gerrit-Reviewer: Kudu Jenkins Gerrit-HasComments: Yes
[kudu-CR] log block manager: use move semantics to fill in the block map
Hello Dan Burkert, Adar Dembo, I'd like you to do a code review. Please visit http://gerrit.cloudera.org:8080/8008 to review the following change. Change subject: log_block_manager: use move semantics to fill in the block map .. log_block_manager: use move semantics to fill in the block map This avoids an extra ref-count increment/decrement while loading the block map during startup. This substantially improves startup time. This shaved off another few percent from LBM startup time. On a real host with 11M blocks: Before: I0907 17:20:42.277474 10021 fs_manager.cc:335] Time spent opening block manager: real 14.348s user 0.000s sys 0.001s After: I0907 17:27:13.600186 14550 fs_manager.cc:335] Time spent opening block manager: real 13.401s user 0.000s sys 0.002s According to the benchmark: Before: I0907 17:15:50.645310 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.570s user 0.034s sys 0.001s I0907 17:15:52.195543 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.550s user 0.037s sys 0.001s I0907 17:15:53.755209 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.560s user 0.037s sys 0.001s I0907 17:15:55.263762 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.509s user 0.038s sys 0.001s I0907 17:15:56.818748 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.555s user 0.037s sys 0.001s I0907 17:15:58.379680 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.561s user 0.036s sys 0.001s I0907 17:15:59.913751 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.534s user 0.038s sys 0.000s I0907 17:16:01.461668 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.548s user 0.037s sys 0.001s I0907 17:16:03.020823 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.559s user 0.037s sys 0.001s I0907 17:16:04.549747 20302 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.529s user 0.035s sys 0.001s After: I0907 17:07:40.230087 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.495s user 0.032s sys 0.001s I0907 17:07:41.642105 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.412s user 0.044s sys 0.001s I0907 17:07:43.082852 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.441s user 0.040s sys 0.001s I0907 17:07:44.512603 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.430s user 0.043s sys 0.000s I0907 17:07:45.956848 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.444s user 0.041s sys 0.000s I0907 17:07:47.386735 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.430s user 0.037s sys 0.002s I0907 17:07:48.787317 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.401s user 0.041s sys 0.002s I0907 17:07:50.244407 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.457s user 0.038s sys 0.001s I0907 17:07:51.682209 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.438s user 0.037s sys 0.001s I0907 17:07:53.116209 9782 log_block_manager-test.cc:799] Time spent reopening block manager: real 1.434s user 0.037s sys 0.000s Change-Id: I8fe65087585e118f91320ec39f537b5f3ad6552f --- M src/kudu/fs/log_block_manager.cc M src/kudu/fs/log_block_manager.h 2 files changed, 12 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/08/8008/1 -- To view, visit http://gerrit.cloudera.org:8080/8008 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8fe65087585e118f91320ec39f537b5f3ad6552f Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Dan Burkert
