Hey Karl, Sure of course, what I have done to investigate the too many open files is use the following java agent
http://file-leak-detector.kohsuke.org/ <http://file-leak-detector.kohsuke.org/> (sadly enough it isnt java 11 compatible yet, thats why I used following fork / PR for the application I am investigating: https://github.com/kohsuke/file-leak-detector/pull/48 <https://github.com/kohsuke/file-leak-detector/pull/48>) You start up the application with the java agent, and then it uses ASM to wrap every opening of a file with some code to be able to debug where it opens a file. I use following setting when starting up the application: -javaagent:/path/to/jar-file.jar=http=19990. After this you can do “curl http://localhost:19999 <http://localhost:19999/>” which gives a list of all the opened files. This contains the snippets of what I already sent! If you need more info, let me know :) Greets, Roy > On 21 Sep 2020, at 14:29, Karl Pauls <karlpa...@gmail.com> wrote: > > Hi Roy, > > I'm not sure what you are using to create these stacktraces and what > they are telling me. Could you try to give some more detail so that I > can reproduce them? > > That said, we do have some way to limit open zip files using the > "felix.cache.filelimit" property (see [0]). I think that might be what > you want - however, if you have a way to reproduce the above and there > are two open files I would still be interested in knowing about it... > > regards, > > Karl > > [0] > http://felix.apache.org/documentation/subprojects/apache-felix-framework/apache-felix-framework-configuration-properties.html#framework-configuration-properties > > On Fri, Sep 18, 2020 at 4:26 PM Roy Teeuwen <r...@teeuwen.be> wrote: >> >> Hey Devs, >> >> I am investigating why our application is sometimes going over the open file >> limit of the linux servers we are on, and one of the things I have noticed >> so far is that felix actually has an open file for every bundle twice! An >> example here is given. It seems that the java.util.zip.ZipFile always has >> two open files. >> >> Is this something that is known? Could there be anything done to fix this so >> that it’s at least only one open file per bundle? Our application has 800 >> bundles => 1600 open files goes to the bundles alone, not counting the >> embedded jars that also count as an extra open file (which would result to >> 2200 open files if you cound those too) >> >> Greets, >> Roy >> >> #1648 /opt/application/launchpad/felix/bundle339/version0.0/bundle.jar by >> thread:main on Fri Sep 18 16:09:01 CEST 2020 >> at >> java.base/java.io.RandomAccessFile.<init>(RandomAccessFile.java:216) >> at java.base/java.util.zip.ZipFile$Source.<init>(ZipFile.java:1271) >> at java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:1237) >> at >> java.base/java.util.zip.ZipFile$CleanableResource.<init>(ZipFile.java:727) >> at >> java.base/java.util.zip.ZipFile$CleanableResource.get(ZipFile.java:844) >> at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:247) >> at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:177) >> at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:191) >> at >> org.apache.felix.framework.util.SecureAction.openZipFile(SecureAction.java:651) >> at >> org.apache.felix.framework.util.WeakZipFileFactory$WeakZipFile.<init>(WeakZipFileFactory.java:169) >> at >> org.apache.felix.framework.util.WeakZipFileFactory$WeakZipFile.<init>(WeakZipFileFactory.java:151) >> at >> org.apache.felix.framework.util.WeakZipFileFactory.create(WeakZipFileFactory.java:78) >> at >> org.apache.felix.framework.cache.JarRevision.<init>(JarRevision.java:83) >> at >> org.apache.felix.framework.cache.BundleArchive.createRevisionFromLocation(BundleArchive.java:793) >> at >> org.apache.felix.framework.cache.BundleArchive.reviseInternal(BundleArchive.java:480) >> at >> org.apache.felix.framework.cache.BundleArchive.<init>(BundleArchive.java:215) >> at >> org.apache.felix.framework.cache.BundleCache.getArchives(BundleCache.java:432) >> at org.apache.felix.framework.Felix.init(Felix.java:787) >> at org.apache.felix.framework.Felix.init(Felix.java:641) >> at org.apache.sling.launchpad.base.impl.Sling.init(Sling.java:347) >> at org.apache.sling.launchpad.base.impl.Sling.<init>(Sling.java:212) >> at >> org.apache.sling.launchpad.base.app.MainDelegate$1.<init>(MainDelegate.java:180) >> at >> org.apache.sling.launchpad.base.app.MainDelegate.start(MainDelegate.java:180) >> at org.apache.sling.launchpad.app.Main.doStart(Main.java:411) >> at org.apache.sling.launchpad.app.Main.doStart(Main.java:347) >> at org.apache.sling.launchpad.app.Main.main(Main.java:135) >> >> >> #1370 /opt/application/launchpad/felix/bundle339/version0.0/bundle.jar by >> thread:main on Fri Sep 18 16:09:01 CEST 2020 >> at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:178) >> at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:191) >> at >> org.apache.felix.framework.util.SecureAction.openZipFile(SecureAction.java:651) >> at >> org.apache.felix.framework.util.WeakZipFileFactory$WeakZipFile.<init>(WeakZipFileFactory.java:169) >> at >> org.apache.felix.framework.util.WeakZipFileFactory$WeakZipFile.<init>(WeakZipFileFactory.java:151) >> at >> org.apache.felix.framework.util.WeakZipFileFactory.create(WeakZipFileFactory.java:78) >> at >> org.apache.felix.framework.cache.JarRevision.<init>(JarRevision.java:83) >> at >> org.apache.felix.framework.cache.BundleArchive.createRevisionFromLocation(BundleArchive.java:793) >> at >> org.apache.felix.framework.cache.BundleArchive.reviseInternal(BundleArchive.java:480) >> at >> org.apache.felix.framework.cache.BundleArchive.<init>(BundleArchive.java:215) >> at >> org.apache.felix.framework.cache.BundleCache.getArchives(BundleCache.java:432) >> at org.apache.felix.framework.Felix.init(Felix.java:787) >> at org.apache.felix.framework.Felix.init(Felix.java:641) >> at org.apache.sling.launchpad.base.impl.Sling.init(Sling.java:347) >> at org.apache.sling.launchpad.base.impl.Sling.<init>(Sling.java:212) >> at >> org.apache.sling.launchpad.base.app.MainDelegate$1.<init>(MainDelegate.java:180) >> at >> org.apache.sling.launchpad.base.app.MainDelegate.start(MainDelegate.java:180) >> at org.apache.sling.launchpad.app.Main.doStart(Main.java:411) >> at org.apache.sling.launchpad.app.Main.doStart(Main.java:347) >> at org.apache.sling.launchpad.app.Main.main(Main.java:135) > > > > -- > Karl Pauls > karlpa...@gmail.com
