Jagadish created SAMZA-972:
------------------------------
Summary: Add holistic memory monitoring for SamzaContainer
Key: SAMZA-972
URL: https://issues.apache.org/jira/browse/SAMZA-972
Project: Samza
Issue Type: Bug
Reporter: Jagadish
Assignee: Jagadish
This feature introduces physical memory monitoring in SamzaContainer.
Context:
Often memory used by the SamzaContainer process includes
A. JVM Heap memory: This is where all JVM variables live.
B. Native memory: This memory lives out of the JVM heap and is not visible to
the JVM. Examples include used by RocksDb, native libraries that user code
depends on etc.
User jobs could be killed by Yarn if their total memory (A+B) exceeds the
configured maximum of yarn.container.memory.mb.
Currently, while our existing metrics provide visibility into [A] via JMX, we
don't have visibility into [B]. (as it's totally external to the JVM).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
