Github user aledsage commented on a diff in the pull request:
https://github.com/apache/brooklyn-docs/pull/122#discussion_r86764334
--- Diff: guide/ops/troubleshooting/memory-usage.md ---
@@ -0,0 +1,94 @@
+---
+layout: website-normal
+title: "Troubleshooting: Monitoring Memory Usage"
+toc: /guide/toc.json
+---
+
+## Memory Usage
+
+Brooklyn tries to keep in memory as much history of its activity as
possible,
+for displaying through the UI, so it is normal for it to consume as much
memory
+as it can. It uses "soft references" so these objects will be cleared if
needed,
+but **it is not a sign of anything unusual if Brooklyn is using all its
available memory**.
+
+The number of active tasks, CPU usage, thread counts, and
+retention of soft reference objects are a much better indication of load.
+This information can be found by looking in the log for lines containing
+`brooklyn gc`, such as:
+
+ 2016-09-16 16:19:43,337 DEBUG o.a.b.c.m.i.BrooklynGarbageCollector
[brooklyn-gc]: brooklyn gc (before) - using 910 MB / 3.76 GB memory; 98%
soft-reference maybe retention (of 362); 35 threads; tasks: 0 active, 2
unfinished; 31 remembered, 1013 total submitted)
+
+The soft-reference figure is indicative, but the lower this is, the more
+the JVM has decided to get rid of items that were desired to be kept but
optional.
+It only tracks some soft-references (those wrapped in `Maybe`),
+and of course if there are many many such items the JVM will have to get
rid
+of some, so a lower figure does not necessarily mean a problem.
+Typically however if there's no `OutOfMemoryError` (OOME) reported,
+there's no problem.
+
+If you are concerned about memory usage, or doing evaluation on test
environments,
+the following method (in the Groovy console) can be invoked to force the
system to
+reclaim as much memory as possible, including *all* soft references:
+
+
org.apache.brooklyn.util.javalang.MemoryUsageTracker.forceClearSoftReferences()
+
+If things are happy usage should return to a small level. This is quite
disruptive
+to the system however so use with care.
+
+The above method can also be configured to run automatically when memory
usage
+is detected to hit a certain level. That can be useful if external
policies are
+being used to warn on high memory usage, and you want to keep some
headroom.
+Many JVM references discourage interfering with its garbage collector,
however,
+so use with care and study the particular JVM you are using.
+See the class `BrooklynGarbageCollector` for more information.
+
+
+## Investigation of Memory Leaks
+
+Design problems of course can cause memory leaks, and due to the nature of
the
+soft references these can be difficult to notice until they are advanced.
+If the "soft-reference maybe retention" starts to decrease, that can be
+an early warning.
+
+Common problems such as runaway tasks and cyclic dependent configuration
will often
+show their own log errors, so also look for these if there is a
performance or memory problem.
+
+You should also note the task counts in the `brooklyn gc` messages
described above,
+and if there are an exceptional number of tasks or tasks are not clearing,
+other log messages will describe what is happening, and the in-product task
+view can indicate issues. `jstack` can also be useful if it is a task
problem.
--- End diff --
Also worth recommending the use of `jmap -histo:live <pid>` to see how many
live instances of each type there are.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
