Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/13606 )
Change subject: KUDU-2836: Release memory to OS periodically ...................................................................... Patch Set 6: (7 comments) http://gerrit.cloudera.org:8080/#/c/13606/6/src/kudu/integration-tests/memory_gc-itest.cc File src/kudu/integration-tests/memory_gc-itest.cc: http://gerrit.cloudera.org:8080/#/c/13606/6/src/kudu/integration-tests/memory_gc-itest.cc@55 PS6, Line 55: void GetMemorySize(ExternalTabletServer* ets, MemoryType type, int64_t* value) { This is OK, but I wonder if it would be easier to fetch /metrics?metrics=tcmalloc_pageheap_free_bytes,generic_current_allocated_bytes and parse it with RapidJSON? I think that'll give you the same info but be less fragile (and easier to read) than regex. Having this function just give two out-parameters for the two types of memory would also probably simplify it http://gerrit.cloudera.org:8080/#/c/13606/6/src/kudu/integration-tests/memory_gc-itest.cc@87 PS6, Line 87: OverHeadPercent nit: rename to GetOverheadRatio (a ratio is between 0 and 1, but a percent is between 0 and 100) http://gerrit.cloudera.org:8080/#/c/13606/6/src/kudu/integration-tests/memory_gc-itest.cc@138 PS6, Line 138: e typo http://gerrit.cloudera.org:8080/#/c/13606/6/src/kudu/integration-tests/memory_gc-itest.cc@167 PS6, Line 167: // Limit memory usage once a tserver. why do we need to test this once per tserver? I think just running this test once (eg limit tserver 1) is sufficient http://gerrit.cloudera.org:8080/#/c/13606/3/src/kudu/server/server_base.cc File src/kudu/server/server_base.cc: http://gerrit.cloudera.org:8080/#/c/13606/3/src/kudu/server/server_base.cc@211 PS3, Line 211: DEFINE_uint64(gc_tcmalloc_memory_interval_seconds, 3600, is once an hour often enough? from your graphs you uploaded on the JIRA it looked like the memory can grow much faster than this. I was thinking every 30 seconds or so would be OK http://gerrit.cloudera.org:8080/#/c/13606/3/src/kudu/server/server_base.cc@682 PS3, Line 682: gc_interval = MonoDelta::FromSeconds(FLAGS_gc_tcmalloc_memory_interval_seconds > 0 : ? FLAGS_gc_tcmalloc_memory_interval_seconds : 60); : if (FLAGS_gc_tcmalloc_memory_interval_seconds > 0) I think this could be clearer if you rename 'gc_interval' to 'check_interval' and add a comment like: // If GC is disabled, wake up every 30 seconds anyway to recheck the value of the flag. check_interval = FLAGS_gc_tcmalloc_memory_interval_secs > 0 ? FLAGS_gc_tcmalloc_memory_interval_secs : 60; http://gerrit.cloudera.org:8080/#/c/13606/6/src/kudu/server/server_base.cc File src/kudu/server/server_base.cc: http://gerrit.cloudera.org:8080/#/c/13606/6/src/kudu/server/server_base.cc@211 PS6, Line 211: DEFINE_uint64(gc_tcmalloc_memory_interval_seconds, 3600, Let's default this to 30 seconds or something. Otherwise I think a lot of memory can accumulate. -- To view, visit http://gerrit.cloudera.org:8080/13606 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibeb3529ed53c250f69e5faf5b2a067b2abce06c0 Gerrit-Change-Number: 13606 Gerrit-PatchSet: 6 Gerrit-Owner: Yingchun Lai <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Grant Henke <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Todd Lipcon <[email protected]> Gerrit-Reviewer: Yingchun Lai <[email protected]> Gerrit-Comment-Date: Mon, 17 Jun 2019 16:43:56 +0000 Gerrit-HasComments: Yes
