Adar Dembo has posted comments on this change. Change subject: Simplify MemTracker and move process throttling elsewhere ......................................................................
Patch Set 3: (11 comments) http://gerrit.cloudera.org:8080/#/c/6620/3/src/kudu/integration-tests/raft_consensus-itest.cc File src/kudu/integration-tests/raft_consensus-itest.cc: PS3, Line 2254: since we can get accurate process memory : // usage statistic I presume you tested against the kNumOps of 10000, and this new value made sense? http://gerrit.cloudera.org:8080/#/c/6620/3/src/kudu/server/default-path-handlers.cc File src/kudu/server/default-path-handlers.cc: Line 149: *output << "<h1>Total memory usage</h1>\n"; "Total" suggests that the per-subsystem stuff should add up to it. Perhaps "Process memory usage" would be more precise? Line 152: HumanReadableNumBytes::ToString(process_memory::CurrentConsumption())); Maybe add a warning if !TCMALLOC_ENABLED that this isn't accurate? http://gerrit.cloudera.org:8080/#/c/6620/3/src/kudu/tablet/multi_column_writer.cc File src/kudu/tablet/multi_column_writer.cc: Line 89: LOG(INFO) << "Opened CFile writers for " << cfile_writers_.size() << " column(s)"; Heh, got tired of this output? http://gerrit.cloudera.org:8080/#/c/6620/3/src/kudu/util/mem_tracker.cc File src/kudu/util/mem_tracker.cc: PS3, Line 229: // TODO: This might leave us with an allocated resource that we can't use. Do we need : // to adjust the consumption of the query tracker to stop the resource from never : // getting used by a subsequent TryConsume()? Probably irrelevant to us. Line 240: Consume(-bytes); Technically this can fail, yet we drop the failure on the ground. I wonder if we'd be better off just not allowing Consume(negative) and Release(negative). Line 251: process_memory::MaybeGCAfterRelease(bytes); Maybe this new bit should be documented in the header somewhere? Line 264: return CheckLimitExceeded(); Could we just remove one of these two variants if they're identical? http://gerrit.cloudera.org:8080/#/c/6620/3/src/kudu/util/mem_tracker.h File src/kudu/util/mem_tracker.h: PS3, Line 149: LogUsage() Not relevant anymore. Plus, 'id' is actually used for more than just cosmetic stuff. http://gerrit.cloudera.org:8080/#/c/6620/3/src/kudu/util/process_memory.cc File src/kudu/util/process_memory.cc: Line 20: #include <sys/resource.h> > warning: #includes are not sorted properly [llvm-include-order] Nit: this should be in its own group ahead of gflags/gperftools since it's a "real" system include and not a "project" system include. Line 141: // Nothing to do if not using tcmalloc. Maybe this should be moved up into MaybeGCAfterRelease() so we can avoid the increment on g_released_memory_since_gc? -- To view, visit http://gerrit.cloudera.org:8080/6620 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Id16bad7d9a29a83e820a38e9d703811391cffe90 Gerrit-PatchSet: 3 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Todd Lipcon <t...@apache.org> Gerrit-Reviewer: Adar Dembo <a...@cloudera.com> Gerrit-Reviewer: David Ribeiro Alves <davidral...@gmail.com> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Tidy Bot Gerrit-Reviewer: Todd Lipcon <t...@apache.org> Gerrit-HasComments: Yes
