Todd Lipcon has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/9263 )
Change subject: Make tcmalloc heap sampling more useful ...................................................................... Make tcmalloc heap sampling more useful Previously our /pprof/heap path required that the process be started with lifetime heap profiling enabled. This mode of operation is very slow so we would never be able to use it in production. Even in tests it's of limited utility. The alternative is tcmalloc heap _sampling_ which only records periodic allocations. According to recent mailing list threads, Google runs most of their binaries with this enabled in production, so it's designed to be very low overhead and potentially useful for us in the field. This doesn't go so far as to enable it by default, but it does adjust the /pprof/heap endpoint to dump the sampled allocations instead of starting a profile. Sampling can be enabled using either the tcmalloc env variable (TCMALLOC_SAMPLE_PARAMETER) or a new command line flag --heap-sample-every-n-bytes. I tested this manually by running a kudu tserver with loadgen and sampling set to 500kb. I then used pprof to grab the heap sample information which looked like I expected it to (most of the memory from MRS) Change-Id: I939c2f01f17ceb4b9520bb566f66463952b2a255 Reviewed-on: http://gerrit.cloudera.org:8080/9263 Tested-by: Kudu Jenkins Reviewed-by: Will Berkeley <wdberke...@gmail.com> --- M docs/troubleshooting.adoc M src/kudu/server/pprof_path_handlers.cc M src/kudu/util/flags.cc 3 files changed, 95 insertions(+), 26 deletions(-) Approvals: Kudu Jenkins: Verified Will Berkeley: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/9263 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I939c2f01f17ceb4b9520bb566f66463952b2a255 Gerrit-Change-Number: 9263 Gerrit-PatchSet: 4 Gerrit-Owner: Todd Lipcon <t...@apache.org> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon <t...@apache.org> Gerrit-Reviewer: Will Berkeley <wdberke...@gmail.com>
