Adar Dembo has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14555 )
Change subject: KUDU-2977 Sharding block map to speed up tserver startup ...................................................................... KUDU-2977 Sharding block map to speed up tserver startup Separate LogBlockManager's block lock and container lock, and sharding block map. Tablet server with multiple data directories cost much time to startup, there is a thread for each data directory, each of them should acquire a global lock to add blocks found in its own data directory to the global block map, so there is a queue to acquire lock that may cost much time. The same when remove a batch of blocks. After bunch of benchmarks, we found pretty stable improvment of this patch, time saving much more when block count increasing (reduce about 20% startup time when 10,000,000 blocks on 8 data directories). Sharding count is 8, which has the best performance after comparing with 1, 2, 4, 8, 16, 32, 64 and 128. Change-Id: If0d5c13e051a2c1d6cfd1c9ad7db8a3cd195459d Reviewed-on: http://gerrit.cloudera.org:8080/14555 Reviewed-by: Adar Dembo <[email protected]> Tested-by: Kudu Jenkins --- M src/kudu/fs/log_block_manager-test.cc M src/kudu/fs/log_block_manager.cc M src/kudu/fs/log_block_manager.h 3 files changed, 153 insertions(+), 103 deletions(-) Approvals: Adar Dembo: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/14555 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: If0d5c13e051a2c1d6cfd1c9ad7db8a3cd195459d Gerrit-Change-Number: 14555 Gerrit-PatchSet: 7 Gerrit-Owner: Yingchun Lai <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai <[email protected]>
