Hello Kudu Jenkins, Andrew Wong, Grant Henke, Greg Solovyev, Bankim Bhavsar, Todd Lipcon,
I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/15971 to look at the new patch set (#9). Change subject: [master] cache for table locations ...................................................................... [master] cache for table locations This patch introduces a cache for table locations in catalog manager. When running with 48 concurrent client threads, the performance of CatalogManager::GetTableLocations() method improved about 100% when the cache is enabled. A smaller 14% improvement is observed for GetTableLocations RPC when running with the same number of concurrent client threads and cache enabled. The test results are below. I'm planning to add these test scenarios into kudu/scripts/benchmarks.sh in a follow up patch. ======================================================================== After this patch with 128MByte cache enabled: table_locations-itest \ --gtest_filter=TableLocationsTest.GetTableLocationsBenchmarkFunctionCall \ --benchmark_num_threads=48 \ --table_locations_cache_capacity_mb=128 GetTableLocations function call: 504187.6 req/sec Before this patch: table_locations-itest \ --gtest_filter=TableLocationsTest.GetTableLocationsBenchmarkFunctionCall \ --benchmark_num_threads=48 GetTableLocations function call: 252443.4 req/sec ======================================================================== After this patch with 128MByte cache enabled: table_locations-itest \ --gtest_filter=TableLocationsTest.GetTableLocationsBenchmark \ --rpc_num_service_threads=32 \ --benchmark_num_threads=48 \ --table_locations_cache_capacity_mb=128 GetTableLocations RPC: 40033.4 req/sec Stats on GetTableLocations RPC (times in microseconds): Count: 200167 Mean: 155.144 Percentiles: 0% (min) = 45 25% = 125 50% (med) = 147 75% = 177 95% = 223 99% = 314 99.9% = 652 99.99% = 1928 100% (max) = 5096 Before this patch: table_locations-itest \ --gtest_filter=TableLocationsTest.GetTableLocationsBenchmark \ --rpc_num_service_threads=32 \ --benchmark_num_threads=48 GetTableLocations RPC: 34981 req/sec Stats on GetTableLocations RPC (times in microseconds): Count: 174905 Mean: 249.308 Percentiles: 0% (min) = 64 25% = 197 50% (med) = 231 75% = 284 95% = 386 99% = 556 99.9% = 980 99.99% = 2400 100% (max) = 6113 Change-Id: I7d2a4771ddc455d92a1da00db91c555a21151a23 --- M src/kudu/integration-tests/table_locations-itest.cc M src/kudu/master/CMakeLists.txt M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master_service.cc A src/kudu/master/table_locations_cache.cc A src/kudu/master/table_locations_cache.h A src/kudu/master/table_locations_cache_metrics.cc A src/kudu/master/table_locations_cache_metrics.h 9 files changed, 955 insertions(+), 34 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/71/15971/9 -- To view, visit http://gerrit.cloudera.org:8080/15971 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7d2a4771ddc455d92a1da00db91c555a21151a23 Gerrit-Change-Number: 15971 Gerrit-PatchSet: 9 Gerrit-Owner: Alexey Serbin <aser...@cloudera.com> Gerrit-Reviewer: Alexey Serbin <aser...@cloudera.com> Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com> Gerrit-Reviewer: Bankim Bhavsar <ban...@cloudera.com> Gerrit-Reviewer: Grant Henke <granthe...@apache.org> Gerrit-Reviewer: Greg Solovyev <gsolov...@cloudera.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Todd Lipcon <t...@apache.org>