Hi Peter, Thanks for the follow-up. Yup, I agree with the relationship between Xmx/Xms and UseCGroupMemoryLimitForHeap/MaxRAMFraction.
For the MaxRAMFraction=1 though, it seems that it does leave at least some room for off-heap memory that said, that may not be enough for NiFi and it's normal use-cases. Has anyone run the Native Memory Tracker[1] on a "real-world" system? It gives a nice dump of where/how the JVM is using memory[2]. [1] https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/tooldescr007.html [2] http://trustmeiamadeveloper.com/2016/03/18/where-is-my-memory-java/ Joe On Fri, Aug 31, 2018 at 5:10 AM Peter Wilcsinszky < [email protected]> wrote: > Hi, > > I haven't done extensive research in this area but ran through the > articles and also found another one [1]. From what I understand > UseCGroupMemoryLimitForHeap is just the dynamic version of setting memory > limits manually using Xmx and Xms which is currently done by the NiFi start > script explicitly. In an environment where it should be done in a more > dynamic fashion the UseCGroupMemoryLimitForHeap with proper MaxRAMFraction > should be used but for caveats check the comments here: [1] and here: [2] > (My understanding: MaxRAMFraction=1 considered to be unsafe, > MaxRAMFraction=2 leaves half the memory unused) > > [1] https://banzaicloud.com/blog/java-resource-limits/ > [2] > https://stackoverflow.com/questions/49854237/is-xxmaxramfraction-1-safe-for-production-in-a-containered-environment > > > On Thu, Aug 30, 2018 at 7:54 PM Joe Percivall <[email protected]> > wrote: > >> Hey everyone, >> >> I was recently searching for a best practice guide for running a >> production instance of Apache NiFi within a Docker container and couldn't >> find anything specific other than the normal guidance for best practices of >> a high-performance instance[1]. I did expand my search for best practices >> on running the JVM within a container and found a couple good >> articles[2][3]. The first of which explains why the JVM will take up more >> than is set via "Xmx" and the second is about 2 JVM options which were >> backported from Java 9 to JDK 8u131 specifically for configuring the JVM >> heap for running in a "VM". >> >> So with that, a couple questions: >> 1: Does anyone have any best practices or lessons learned specifically >> for running NiFi in a container? >> 2: "UseCGroupMemoryLimitForHeap" and "MaxRAMFraction" are technically >> "Experimental VM Options", has anyone used them in practice? >> >> [1] >> https://community.hortonworks.com/articles/7882/hdfnifi-best-practices-for-setting-up-a-high-perfo.html >> >> [2] >> https://developers.redhat.com/blog/2017/04/04/openjdk-and-containers/#more-433899 >> [3] >> https://blog.csanchez.org/2017/05/31/running-a-jvm-in-a-container-without-getting-killed/ >> >> Thanks, >> Joe >> -- >> *Joe Percivall* >> linkedin.com/in/Percivall >> e: [email protected] >> > -- *Joe Percivall* linkedin.com/in/Percivall e: [email protected]
