Hi Mircea, G1 has a time limit to collect objects not used in the young generation and if the time limit (200ms by default) is not reached it may continue with collecting unused objects from the old generation. Depends on your context, but I saw applications with small heap where almost all heap was used by young gen when the heap was under 4gb.
You are using just a 2gb heap. Try measuring with parallelgc, g1 needs more memory to store metadata. You could use that memory for your application instead and the gc pauses could be acceptable. Depends on your context. This talk may be useful to look at https://www.youtube.com/watch?v=K_vuBOjJtxk if you would like to get tips to tune G1GC - the presenter is author of the gceasy.io Regards, Zdenek Henek On Thu, Jul 31, 2025 at 10:07 AM Mircea Butmalai <mircea.butma...@radcom.ro> wrote: > Hi, > > Catalina out has captured java options as follows: > > 25-Jul-2025 13:05:57.337 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Server version name: > Apache Tomcat/10.1.43 > 25-Jul-2025 13:05:57.338 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Server built: > Jul 1 2025 21:30:20 UTC > 25-Jul-2025 13:05:57.339 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Server version > number: 10.1.43.0 > 25-Jul-2025 13:05:57.339 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log OS Name: > Linux > 25-Jul-2025 13:05:57.339 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log OS Version: > 5.14.0-533.el9.x86_64 > 25-Jul-2025 13:05:57.339 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Architecture: > amd64 > 25-Jul-2025 13:05:57.339 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Java Home: > /opt/ivroro/app/jdk-21.0.7+6 > 25-Jul-2025 13:05:57.339 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log JVM Version: > 21.0.7+6-LTS > 25-Jul-2025 13:05:57.339 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor: > Eclipse Adoptium > 25-Jul-2025 13:05:57.339 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE: > /opt/ivroro/app/apache-tomcat-10.1.43 > 25-Jul-2025 13:05:57.339 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME: > /opt/ivroro/app/apache-tomcat-10.1.43 > 25-Jul-2025 13:05:57.359 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: > -Djava.util.logging.config.file=/opt/ivroro/app/apache-tomcat/conf/logging.properties > 25-Jul-2025 13:05:57.359 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -Xmixed > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -Xms512m > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -Xmx2G > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -Xss1024k > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -XX:MetaspaceSize=256m > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -XX:MaxMetaspaceSize=512m > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -XX:+UseG1GC > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -Dsun.net.inetaddr.ttl=1800 > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -Dsun.net.inetaddr.negative.ttl=0 > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -Dnetworkaddress.cache.ttl=1800 > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -Dnetworkaddress.cache.negative.ttl=0 > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -Dcatalina.controlport=8005 > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -Dcatalina.httpport=8080 > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -Dcatalina.httpsport=8443 > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -Dcatalina.ajpport=8009 > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -Divroro.config.file.path=/opt/ivroro/config/ivroro.properties > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: > -Divroro.log4j.file.path=/opt/ivroro/config/ivroro-log4j.properties > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -Djdk.tls.ephemeralDHKeySize=2048 > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -Dsun.io.useCanonCaches=false > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: --add-opens=java.base/java.lang=ALL-UNNAMED > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: --add-opens=java.base/java.lang.reflect=ALL-UNNAMED > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: --add-opens=java.base/java.io=ALL-UNNAMED > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: --add-opens=java.base/java.util=ALL-UNNAMED > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: --add-opens=java.base/java.util.concurrent=ALL-UNNAMED > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -Dcatalina.base=/opt/ivroro/app/apache-tomcat > 25-Jul-2025 13:05:57.360 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -Dcatalina.home=/opt/ivroro/app/apache-tomcat > 25-Jul-2025 13:05:57.361 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Command line > argument: -Djava.io.tmpdir=/opt/ivroro/app/apache-tomcat/temp > > > So as can be seen the G1 collector is in use (which I presume is the > default anyway). > > Question: > Can you confirm that if this collector is in use and no counter for FULL > GC is incremented then no object from OLD SPACE is actually collected? > The G1 collector seems very complicated even from it's documentation. > > > Thanks, > Mircea Butmalai > > > ________________________________ > From: Christopher Schultz <ch...@christopherschultz.net> > Sent: Thursday, July 31, 2025 1:27 AM > To: Tomcat Users List <users@tomcat.apache.org>; Mircea Butmalai < > mircea.butma...@radcom.ro> > Subject: Re: memory leak analysis > > Mircea, > > On 7/29/25 3:33 PM, Mircea Butmalai wrote: > > I have created a public repo on github with the following URL: > > mirceabutmalai/test-docs: test-docs< > https://github.com/mirceabutmalai/test-docs> > > > > Inside you have 2 png files which are referred from initial email and > also the complete jstat log file obtained during test. > > It would be much better to use verbose GC logging and capture that. It's > also pretty important when analyzing Java heap memory to include your > heap configuration along with any other information you have. > > For example, we don't even know which garbage collector you are using. > > You are barely asking Tomcat to do anything over that 3 day period. Zero > GC events would not be surprising depending upon your heap settings. > > -chris > > > ________________________________ > > From: Gregg, John E. <john.e.gr...@wellsfargo.com.INVALID> > > Sent: Tuesday, July 29, 2025 9:09 PM > > To: Tomcat Users List <users@tomcat.apache.org> > > Subject: RE: memory leak analysis > > > > Mircea, > > > > If your employer allows it, upload your GC logs to > https://gceasy.io/gc-index.jsp. One benefit of this tool over some of > the others is that it actually gives you qualitative feedback on what might > be wrong, if anything. > > > > Thanks > > > > > > > > From: Mircea Butmalai <mircea.butma...@radcom.ro> > > Sent: Tuesday, July 29, 2025 8:21 AM > > To: Tomcat Users List <users@tomcat.apache.org> > > Subject: memory leak analysis > > > > > > > > Hello, > > > > I have analyzed java memory heap for approximately default tomcat > 10.1.43 installation. > > Approximately default tomcat 10.1.43 installation means: > > > > * Installation of tomcat is done on CentOS 9 with uname output as: > > Linux centos9dev1.local 5.14.0-533.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri > Nov 22 15:05:06 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux > > > > > > * Java environment is Eclipse Temurin JDK 21.0.7 > > > > * Apache Tomcat 10.1.43 archive tar.gz obtained from official > tomcat site and installed under special folder inside /opt specific to our > project > > > > * No .bat files in tomcat bin folder > > > > * no examples and docs webapps, only default ROOT, manager and > host-manager webapps are delivered from official tomcat archive. No other > custom webapps are present in this heap memory test. > > > > * tomcat file conf/tomcat-users.xml contains user with roles > manager-script, manager-status, manager-jmx, manager-gui > > > > * Custom startup / shutdown / check bash scripts that basically > launch catalina.sh with custom java options. > > > > * The check script verifies tomcat process and if ok it checks > using manager URL /manager/text/serverinfo in order to decide if tomcat is > ok. The client http access is done using system application curl. > > > > * These custom scripts launch process under custom created system > user and all tomcat and java files are owned by this custom system user > > > > The heap memory analysis was done as follows: > > > > * Due to the fact that server info manager URL is already accessed > by our custom check script at a rate of 0.1 requests per second it was > decided that first test was to inject a bigger rate (5 requests per second) > into tomcat server from this manager URL /manager/text/serverinfo. Only > this requests where inside tomcat. > > > > * Before injecting these requests a jstat process was started in > order to monitor heap memory at a rate of 5s. So we have a text file output > from this java application that offers stats about java memory collections. > > > > * Data was collected for a duration of 265k seconds = aprox. 3 days > > > > Graphical representation of EU (eden space utilization in KB), OU (old > space utilization KB) and MU (metaspace utilization KB) are attached inside > this email. > > > > The questions are: > > > > 1. Why we have a constant increase for OU during this test. > > > > 1. The slope is significant and should be zero during this kind of > test at least after some reasonable time period like some minutes. > > > > 1. It is there a problem? When the slope is going to change to zero? > > > > 1. It seems like there is a very little slope even for MU which > increases from 16793.6 KB at 39k seconds to 16809 KB at 265k seconds. (diff > is 15.4 KB). The increase here is linearly distributed over the time > interval. It is a problem even here? > > > > Thanks, > > Mircea Butmalai > > > > > >
