Thanks for sending those over. I'm digging through this - I *think* I have pinned down a specific reference that stops that classloader from being released. The behaviour for a war/ear deployed in apps/ seems different to deploying in webapps/ - I'm assuming that's what you're doing. If you can confirm, that would help. I'll test out a patch with some bigger .ear files today.
Thanks Jon On Thu, Dec 12, 2019 at 9:08 PM Jonathan Gallimore < [email protected]> wrote: > The mailing list seems to be stripping off your images - can you resend > and include my email address as well? I'd be keen to dig into this a little > more. > > Thanks > > Jon > > On Thu, Dec 12, 2019 at 9:00 PM Paul Carter-Brown > <[email protected]> wrote: > >> Hi, >> >> The code does seem to use org.apache.openejb.core.TempClassLoader. I can >> see one instance in the heap for every war in my ear. + 1. >> >> The TempClassLoader however still has 100's of strong references to it. >> E.g: >> >> [image: image.png] >> >> [image: image.png] >> >> >> Paul Carter-Brown >> Director >> Jini Guru >> m: +27 (0) 83 442 7179 <+27834427179> >> a: 1st Floor, Golf House, Design Quarter, Cnr. William Nicol and Leslie >> Johannesburg, South Africa >> w: jini.guru e: [email protected] >> >> Disclaimer: This message and/or attachment(s) may contain >> privileged, confidential and/or personal information. If you are not the >> intended recipient you may not disclose or distribute any of >> the information contained within this message. In such case you must >> destroy this message and inform the sender of the error. Jini Guru may not >> accept liability for any errors, omissions, information and viruses >> contained in the transmission of this message. Any opinions, conclusions >> and other information contained within this message not related to Jini >> Guru official business is deemed to be that of the individual only and is >> not endorsed by Jini Guru. >> >> >> >> On Thu, Dec 12, 2019 at 9:29 AM Mark Struberg <[email protected]> >> wrote: >> >>> Hi! >>> >>> All this is just a workaround imo. >>> >>> Afair we (Romain) introduced a temporary throw-away ClassLoader for >>> scanning. That means after doing all the class scans we only keep the >>> extracted information but do not keep the Classes in memory. Doesn't this >>> work anymore? >>> >>> LieGrue, >>> strub >>> >>> >>> > Am 12.12.2019 um 00:45 schrieb Paul Carter-Brown >>> <[email protected]>: >>> > >>> > FYI. Graph of the change in heap size on a prod instance after this >>> change: >>> > >>> > >>> > Paul Carter-Brown >>> > Director >>> > Jini Guru >>> > m: +27 (0) 83 442 7179 >>> > a: 1st Floor, Golf House, Design Quarter, Cnr. William Nicol and >>> Leslie >>> > Johannesburg, South Africa >>> > w: jini.guru e: [email protected] >>> > >>> > Disclaimer: This message and/or attachment(s) may contain privileged, >>> confidential and/or personal information. If you are not the intended >>> recipient you may not disclose or distribute any of the information >>> contained within this message. In such case you must destroy this message >>> and inform the sender of the error. Jini Guru may not accept liability for >>> any errors, omissions, information and viruses contained in the >>> transmission of this message. Any opinions, conclusions and other >>> information contained within this message not related to Jini Guru official >>> business is deemed to be that of the individual only and is not endorsed by >>> Jini Guru. >>> > >>> > >>> > >>> > >>> > On Thu, Dec 12, 2019 at 1:17 AM Paul Carter-Brown >>> <[email protected]> wrote: >>> > Hi Jon, >>> > >>> > I took a look at the source and worked out that I could add my own >>> filters using openejb.additional.exclude. >>> > >>> > I set it to openejb.additional.exclude=com,net,io,org and the JVM heap >>> now sits at around 150MB whereas it would never go below 450MB previously!!! >>> > >>> > Our EJB's and code is all in jars prefixed with guru.jini and hence >>> they are not filtered out and the system works 100%. >>> > >>> > My sense is that there should be an easier way to specify what jars >>> and/or packages to process for annotations. I have come across the >>> following settings but can't really work out what fits where: >>> > >>> > openejb.additional.exclude >>> > openejb.additional.include >>> > openejb.deployments.classpath.filter.descriptors >>> > openejb.deployments.classpath.filter.systemapps >>> > openejb.deployments.classpath.include >>> > openejb.deployments.classpath.exclude >>> > openejb.deployments.package.include >>> > openejb.deployments.package.exclude >>> > >>> > >>> > P.S. Perhaps also add these to the standard exclusion list as they are >>> common and yet won't need to be processed for annotations I guess? >>> > com.amazonaws >>> > com.fasterxml >>> > com.google >>> > com.hazelcast >>> > io.grpc >>> > io.netty >>> > org.docx4j >>> > org.mongodb >>> > org.rocksdb >>> > org.asynchttpclient >>> > org.apache >>> > org.aspectj >>> > >>> > But then maybe some are too broad and I don't really understand what >>> the annotation index/cache is really needed for at runtime? Can it not be >>> lazy loaded or discarded if unused? Just thinking out loud here :-( Seems >>> like the cache uses a significant amount of heap when considering the drive >>> towards tiny micro services. >>> > >>> > Paul Carter-Brown >>> > Director >>> > Jini Guru >>> > m: +27 (0) 83 442 7179 >>> > a: 1st Floor, Golf House, Design Quarter, Cnr. William Nicol and >>> Leslie >>> > Johannesburg, South Africa >>> > w: jini.guru e: [email protected] >>> > >>> > Disclaimer: This message and/or attachment(s) may contain privileged, >>> confidential and/or personal information. If you are not the intended >>> recipient you may not disclose or distribute any of the information >>> contained within this message. In such case you must destroy this message >>> and inform the sender of the error. Jini Guru may not accept liability for >>> any errors, omissions, information and viruses contained in the >>> transmission of this message. Any opinions, conclusions and other >>> information contained within this message not related to Jini Guru official >>> business is deemed to be that of the individual only and is not endorsed by >>> Jini Guru. >>> > >>> > >>> > >>> > >>> > On Wed, Dec 11, 2019 at 7:18 PM Paul Carter-Brown >>> <[email protected]> wrote: >>> > Hi Jon, >>> > >>> > Unfortunately the snapshot behaves exactly the same way >>> > >>> > Paul Carter-Brown >>> > Director >>> > Jini Guru >>> > m: +27 (0) 83 442 7179 >>> > a: 1st Floor, Golf House, Design Quarter, Cnr. William Nicol and >>> Leslie >>> > Johannesburg, South Africa >>> > w: jini.guru e: [email protected] >>> > >>> > Disclaimer: This message and/or attachment(s) may contain privileged, >>> confidential and/or personal information. If you are not the intended >>> recipient you may not disclose or distribute any of the information >>> contained within this message. In such case you must destroy this message >>> and inform the sender of the error. Jini Guru may not accept liability for >>> any errors, omissions, information and viruses contained in the >>> transmission of this message. Any opinions, conclusions and other >>> information contained within this message not related to Jini Guru official >>> business is deemed to be that of the individual only and is not endorsed by >>> Jini Guru. >>> > >>> > >>> > >>> > >>> > On Wed, Dec 11, 2019 at 3:39 PM Jonathan Gallimore < >>> [email protected]> wrote: >>> > Hi Paul >>> > >>> > Would you mind trying your application with a recent snapshot? >>> > >>> https://repository.apache.org/content/repositories/snapshots/org/apache/tomee/apache-tomee/8.0.1-SNAPSHOT/ >>> . >>> > I recently updated the exclude list. >>> > >>> > Many thanks >>> > >>> > Jon >>> > >>> > On Wed, Dec 11, 2019 at 12:31 PM Paul Carter-Brown >>> > <[email protected]> wrote: >>> > >>> > > Hi, >>> > > >>> > > I've been trying to lower the memory footprint of an EAR deployed to >>> TomEE >>> > > 8.0.0. >>> > > >>> > > Here is a screenshot of the heap histogram. The total Old Gen is >>> about >>> > > 450MB (after forcing multiple GC's). If I boot TomEE without my EAR >>> then >>> > > the old gen is about 6MB. >>> > > >>> > > [image: Screenshot from 2019-12-11 12-53-12.png] >>> > > >>> > > The entire ear is 140MB zip, most of which is in the ears /lib >>> directory >>> > > which contains libs such as Kafka, hazelcast, apache POI, Google >>> cloud >>> > > APIs, AWS client APIs etc etc. In total our code has about 100 actual >>> > > EJB's. If i remove files from the lib folder in the ear then I can >>> see that >>> > > the memory used by the annotation finder is lowered. >>> > > >>> > > Is there any way I can tell TomEE that it need not bother scanning >>> > > everything in the /lib folder of my EAR for annotations and fulling >>> up the >>> > > heap. I already >>> > > set openejb.deployments.classpath.include=.*jg-arch-core-impl.* to >>> scan >>> > > only the one jar in /lib which does have EJB's in it and it seems to >>> obey >>> > > this property but it doesn't seem to mean that annotation processing >>> is >>> > > skipped for all these other jars in /lib >>> > > >>> > > Thanks! >>> > > >>> > > Paul Carter-Brown >>> > > Director >>> > > Jini Guru >>> > > m: +27 (0) 83 442 7179 <+27834427179> >>> > > a: 1st Floor, Golf House, Design Quarter, Cnr. William Nicol and >>> Leslie >>> > > Johannesburg, South Africa >>> > > w: jini.guru e: [email protected] >>> > > >>> > > Disclaimer: This message and/or attachment(s) may contain >>> > > privileged, confidential and/or personal information. If you are not >>> the >>> > > intended recipient you may not disclose or distribute any of >>> > > the information contained within this message. In such case you must >>> > > destroy this message and inform the sender of the error. Jini Guru >>> may not >>> > > accept liability for any errors, omissions, information and viruses >>> > > contained in the transmission of this message. Any opinions, >>> conclusions >>> > > and other information contained within this message not related to >>> Jini >>> > > Guru official business is deemed to be that of the individual only >>> and is >>> > > not endorsed by Jini Guru. >>> > > >>> > > >>> >>>
