Will Berkeley has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10266 )
Change subject: Add validation and docs on setting block_cache_capacity_mb too high ...................................................................... Add validation and docs on setting block_cache_capacity_mb too high Raising --block_cache_capacity_mb can improve Kudu's performance, but raising it too high (above the memory pressure threshold, which is --memory_pressure_percentage of the hard memory limit) causes constant flushing even if write throughput is low. This patch adds validation of the capacity using a group flag validator, and does not permit Kudu to start if the block cache size can can cause Kudu to get stuck in aggressive flushing mode. It warns if the capacity is large enough (> 50%) of the pressure threshold. I chose 50% as the max recommended block cache size as a percentage of the memory pressure threshold somewhat arbitrarily. It seems safe, but also leaves room for allocating a large block cache. There's no automated tests but I did verify the expected messages appeared in the logs and the server fails to start when --block_cache_capacity_mb is too high. Change-Id: Ia99fa95c578235beb3a1c72a33a22b4d8ff4800c Reviewed-on: http://gerrit.cloudera.org:8080/10266 Reviewed-by: Alexey Serbin <[email protected]> Tested-by: Kudu Jenkins --- M docs/troubleshooting.adoc M src/kudu/cfile/block_cache.cc M src/kudu/integration-tests/client-stress-test.cc M src/kudu/integration-tests/raft_consensus-itest.cc M src/kudu/util/process_memory.cc M src/kudu/util/process_memory.h 6 files changed, 76 insertions(+), 10 deletions(-) Approvals: Alexey Serbin: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/10266 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia99fa95c578235beb3a1c72a33a22b4d8ff4800c Gerrit-Change-Number: 10266 Gerrit-PatchSet: 8 Gerrit-Owner: Will Berkeley <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon <[email protected]> Gerrit-Reviewer: Will Berkeley <[email protected]>
