Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/14743 )
Change subject: KUDU-3001 Multi-thread to load containers in a data directory ...................................................................... Patch Set 12: (3 comments) Hao and Andrew, could you two take a look at this? http://gerrit.cloudera.org:8080/#/c/14743/12//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/14743/12//COMMIT_MSG@32 PS12, Line 32: Block count old version | 1 thread | 5 threads Interesting. Your results suggest that >1 threads on spinning disks are beneficial, so let's try to figure out what a good default value would be. Could you rerun with additional powers-of-2 thread values? Let's try 1, 2, 4, 8, 16, 32... until the results get worse. http://gerrit.cloudera.org:8080/#/c/14743/12/src/kudu/fs/log_block_manager.h File src/kudu/fs/log_block_manager.h: http://gerrit.cloudera.org:8080/#/c/14743/12/src/kudu/fs/log_block_manager.h@261 PS12, Line 261: typedef std::unique_ptr<internal::LogBlockContainerLoadResult> LoadResult; This is only used in one place now; could we remove it to avoid confusion with LogBlockContainerLoadResult? http://gerrit.cloudera.org:8080/#/c/14743/11/src/kudu/fs/log_block_manager.cc File src/kudu/fs/log_block_manager.cc: http://gerrit.cloudera.org:8080/#/c/14743/11/src/kudu/fs/log_block_manager.cc@2077 PS11, Line 2077: dir_result->dead_containers.insert( > Here I aim to _append_ multiple container_results' dead_containers to dir_r Ah, I missed the inner loop. Makes sense. -- To view, visit http://gerrit.cloudera.org:8080/14743 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0721ee4a5a6824db146ba0658e60eec25dd0c65c Gerrit-Change-Number: 14743 Gerrit-PatchSet: 12 Gerrit-Owner: Yingchun Lai <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai <[email protected]> Gerrit-Reviewer: wangning <[email protected]> Gerrit-Comment-Date: Thu, 05 Dec 2019 19:04:22 +0000 Gerrit-HasComments: Yes
