Hello Mark, I briefly want to ask whether the internal discussion about the open JVM file handle when using sendfile/Memory-Mapped-Files resulted in any conclusions?
Thanks in advance! Thomas > -----Ursprüngliche Nachricht----- > Von: Mark Thomas <ma...@apache.org> > Gesendet: Montag, 20. Juni 2022 22:13 > An: users@tomcat.apache.org > Betreff: Re: AW: AW: AW: Filehandle left open when using sendfile > > On 20/06/2022 11:39, Thomas Hoffmann (Speed4Trade GmbH) wrote: > > Hello Mark, > > > > thanks for your reply! > > > >> -----Ursprüngliche Nachricht----- > >> Von: Mark Thomas <ma...@apache.org> > >> Gesendet: Montag, 20. Juni 2022 12:06 > >> An: users@tomcat.apache.org > >> Betreff: Re: AW: AW: Filehandle left open when using sendfile > >> > >> On 16/06/2022 19:58, Thomas Hoffmann (Speed4Trade GmbH) wrote: > >> > >> <snip/> > >> > >>> In the meantime I stumbled upon this bug-Report: > >>> https://bugs.java.com/bugdatabase/view_bug.do?bug_id=4715154 > >>> So maybe the problem lies even deeper. > >>> Similar description here: > >>> https://cemerick.com/blog/2006/08/30/memory-mapping-files-in-java- > >> caus > >>> es-problems.html > >>> > >>> Some ppl suggest to use java.lang.ref.Cleaner or don’t use Memory- > >> Mapped files under Windows. > >>> I don’t know if there are other solutions. > >> > >> Your research looks to be exhaustive. I can't find any better ideas. > >> > >> Using the java.lang.ref.Cleaner looks to be a viable option. We know > >> when the mapped file is no longer being used. However, that requires > >> Java 12 onwards. > >> > >> This is only going to be required if the file locking is an issue. In > >> read-only scenarios or when using an OS other than Windows it won't be > an issue. > >> > >> So, what do we want to do? > >> > >> 1. Disable sendfile for HTTP/2 if running on Windows? > >> > >> 2. Document the potential issues with sendfile + HTTP/2 + Windows if > >> resources are not read-only? > >> > >> 3. Use the JreCompat mechanism to clear the references if possible: > >> - if running on Windows > >> - on all OSes > >> - if enabled via configuration > >> > >> Something else? > >> > >> Mark > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > >> For additional commands, e-mail: users-h...@tomcat.apache.org > > > > I did some further searching on this topic. > > Several posts disregard using java.lang.ref.Cleaner because if the buffer is > used afterwards, it will crash the VM. But if used carefully it works. > > If we use this option, it should be possible to use it appropriately > carefully. > > > About your suggestions: > > 2) Documenting would be helpful, if lock can't be prevented. > > I also found documentation at e.g. > https://docs.oracle.com/javase/9/docs/api/java/nio/channels/FileChannel.h > tml#map-java.nio.channels.FileChannel.MapMode-long-long- > > " The buffer and the mapping that it represents will remain valid > > until > the buffer itself is garbage-collected." > > Which is essentially the problem. Using the Cleaner would clean up the > reference sooner. > > > 3) As JreCompat is a bit risky, enabling via config sounds safe to me. > > JreCompat is perfectly safe. The jdk.internal.misc.Unsafe API is where the > risk is and this is primarily the risk of the crash mentioned above that we > should be able to avoid. > > > Some other (theoretical?) options: > > 4) In an older version of Tomcat native lib there seemed to be a native > Implementation of MMap: https://tomcat.apache.org/tomcat-10.0- > doc/api/org/apache/tomcat/jni/Mmap.html > > I read that this was an alternative to the Java memory mapped > > file. But it was removed in newer versions. Maybe it can be > > resurrected for this case and used if native lib is available(?) > > Sorry, no. We are moving away from the native library. Eventually we will just > use project Panama to wrap OpenSSL. Until then, we are removing > everything that isn't required to support the use of OPenSSl with NIO and > NIO2. > > The primary reason for this is stability. > > > 5) Instead of FileChannel.map maybe a normal ByteBuffer with > > FileChannel.read(buffer) can be used (?) > > That is worth considering. The other sendfile implementations don't use a > memory mapped file. > > I'll start a discussion on the dev list. > > > One remaining question: > > I didn’t find FileChannel.map in the other connectors. Is > Http2AsyncUpgradeHandler the only occurrence? > > In the main code base, yes. There is another usage in the test code but that > is > less of a concern. > > Mark > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
