Ah I think I know!The code is fine, it's just how you use it. Could it be that you are compiling this as a module and also use it as a module (the stack trace implies this)? If you use the code as a "real" module, you have to import explicitely in your module-info the "jdk.unsupported" module, otherwise you won't get access to sun.misc. You may need to add this to the auto-module descriptor or module-info.java.
This must not only be done in tests, Your JAR file MUST declare that it needs "jdk.unsupported" module. If it does not do this CleanerUtil is useless. In contrast to the classpath approach, where there is a default set of modules enabled, in a modulepath you only get what you declare - very simple. In short: You code fails, because the Class.forName() in the Java 9 code will not find "sun.misc.Unsafe", because its shielded by the module system. Because of this it falls back to the Java 8 code, which won't work at all (causing the exceptions). Uwe ----- Uwe Schindler [email protected] ASF Member, Apache Lucene PMC / Committer Bremen, Germany https://lucene.apache.org/ > -----Original Message----- > From: Uwe Schindler <[email protected]> > Sent: Friday, August 7, 2020 5:20 PM > To: 'POI Developers List' <[email protected]>; 'Andreas Beeker' > <[email protected]>; [email protected] > Subject: RE: Java 9+ modules / cleaner > > Hi, > > sorry I did not read all, so you already use my code: > > The exception you see is coming from the fact that the first part for Java 9 > does > not work at all. It looks like the code uses Java 8 variant (see line number > in > exception) instead of the Java 9 variant. > > Can you make a breakpoint here: > https://svn.apache.org/viewvc/poi/trunk/src/java/org/apache/poi/poifs/nio/Cle > anerUtil.java?view=markup#l99 > > And step through the code? It might fall through to the catch block and java 8 > code. > > Uwe > > ----- > Uwe Schindler > Achterdiek 19, D-28357 Bremen > https://www.thetaphi.de > eMail: [email protected] > > > -----Original Message----- > > From: Uwe Schindler <[email protected]> > > Sent: Friday, August 7, 2020 5:06 PM > > To: 'Andreas Beeker' <[email protected]>; [email protected] > > Cc: [email protected] > > Subject: RE: Java 9+ modules / cleaner > > > > Hi, > > > > > I currently try to understand, how to call the Cleaner in Java 14 (or 9+) > > without > > > adding the --add-opens JVM options. > > > > Yeah, your code won't work correctly with Java 9 at all. You may fix it with > > some opens, but still types of internal calsses changed, so its just risky > > (everything is subject to change). > > > > > As you've worked on this in LUCENE-6989, you might have a few hints for > me. > > > > I think you can more or less copy the code from Lucene (use the branch_8x > > version, as Master requires Java 11, so has no Java 8 code anymore, see > > here: > > https://tinyurl.com/y47euqfg). The aproach in Lucene does NOT use > Reflection > > at all, it works with Method Handles. The important thing is: Method handles > > are linked earyl, once you have built the final method handle to invoke the > > cleaner, you can call it without requesting any additional right (security > > manager). In addition you know beforehand if it works at all (it cannot > > throw > > extra reflective exceptions). > > > > The main change in Java 9 is (and this is what's officially "supported" by > > OpenJDK developer): They added a new method to sun.misc.Unsafe (the > legacy > > one), Unsafe#invokeCleaner(ByteBuffer). This class is still in java.base > > and is > > open to public (if you know how to get the singleton) for the time being. To > get > > the singleton, you need reflection and the code must allow to do > setAccessible > > on the getter, but once you have it, it's useable. > > > > To unmap a Bytebuffer, Lucene creates a MethodHandle with some signature > > like "unmap(ByteBuffer b)" complete pre-configued on startup of class > > depending on Java version: > > - In Java 9 and above it uses reflection to get the Unsafe instance (this > requires > > security manager to allow it). Then it looks up the method > > "invokeCleaner(ByteBuffer) and binds it to the unsafe singleton, the final > > methodhandle is casted to me "void unmap(ByteBuffer)" > > - In Java 8 and before it uses more or less the old approach by checking the > > private method to get the Cleaner instance. Finally it calls > > cleaner.clean(). > This > > can also be composed to a MethodHandle with exactly same signature (using > > the famous MethodHandle transformation and bindings, introducing some > null > > checks). The result is also a methodhandle with signature "void > > unmap(ByteBuffer)". > > > > Once all this is done, the methodhandle with platform independent signature > > can be called without any exception handling from any code, so be sure to > keep > > it safe in private final fields fully internal to your implementaion > > (otherwise > it's > > a security issue). > > > > > I've checked the Lucene implementation, but that code is similar to POIs > > > current implementation. [1] > > > As I don't see the Runable interface, I might look at the wrong branch. > > > > That won't work. It's the same approach like the old one, just with other > > class > > types. You cant work around the internal Cleaner interface. This is why > > sun.misc.Unsafe#invokeCleaner() was added. > > > > > Any ideas? > > > > See above. I'd copy the code from Lucene: It's early binding and failsafe, > > once > > you got the MethodHandle. The approach should work with Java 7+. In Java 7 > > there is one additional helper method needed for the methodHandle > regarding > > the null check! > > > > Java 8 is here: https://tinyurl.com/y47euqfg > > Hack for compatibility with Java 7 is here: https://tinyurl.com/y4drev3k > > (this > > adds a "compatibility method" to replace missing "Objects#nonNull(Object)", > > but it's identical otherwise; we added this to make Lucene 5.5 still > > compatible > > with Java 9, long after support ended, because customers need this). > > > > Uwe > > > > > Best wishes, > > > Andi > > > > > > > > > [1] > > > https://github.com/apache/lucene- > > > > > > solr/blob/master/lucene/core/src/java/org/apache/lucene/store/MMapDirecto > > > ry.java#L338 > > > vs. > > > > > > https://svn.apache.org/viewvc/poi/trunk/src/java/org/apache/poi/poifs/nio/Cle > > > anerUtil.java?view=markup#l91 > > > > > > On 09.07.20 00:34, Andreas Beeker wrote: > > > > Hi Dominik, > > > > > > > > the goal is to have no --add-opens or similar jvm arguments. In this > > > > case > we > > > get the following exception: > > > > > > > > [junit] java.lang.reflect.InaccessibleObjectException: Unable to make > > public > > > jdk.internal.ref.Cleaner java.nio.DirectByteBuffer.cleaner() accessible: > module > > > java.base does not "opens java.nio" to module org.apache.poi.poi > > > > [junit] at > > > > > > java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleO > > > bject.java:349) > > > > [junit] at > > > > > > java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleO > > > bject.java:289) > > > > [junit] at > > > > java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:196) > > > > [junit] at > > > java.base/java.lang.reflect.Method.setAccessible(Method.java:190) > > > > [junit] at > > > > > > org.apache.poi.poi/org.apache.poi.poifs.nio.CleanerUtil.unmapHackImpl(Cleane > > > rUtil.java:116) > > > > [junit] at > > > > > > java.base/java.security.AccessController.doPrivileged(AccessController.java:312 > > > ) > > > > [junit] at > > > > > > org.apache.poi.poi/org.apache.poi.poifs.nio.CleanerUtil.<clinit>(CleanerUtil.jav > > > a:77) > > > > [junit] at > > > > > > org.apache.poi.poi/org.apache.poi.poifs.nio.FileBackedDataSource.unmap(FileB > > > ackedDataSource.java:189) > > > > [junit] at > > > > > > org.apache.poi.poi/org.apache.poi.poifs.nio.FileBackedDataSource.lambda$clos > > > e$0(FileBackedDataSource.java:162) > > > > [junit] at > > > java.base/java.util.IdentityHashMap.forEach(IdentityHashMap.java:1356) > > > > [junit] at > > > > > > org.apache.poi.poi/org.apache.poi.poifs.nio.FileBackedDataSource.close(FileBac > > > kedDataSource.java:162) > > > > [junit] at > > > > > > org.apache.poi.poi/org.apache.poi.poifs.filesystem.POIFSFileSystem.close(POIFS > > > FileSystem.java:764) > > > > [junit] at > > > > > > org.apache.poi.poi/org.apache.poi.hpsf.basic.TestWrite.inPlacePOIFSWrite(Test > > > Write.java:539) > > > > > > > > If we use the Runnable aproach mentioned in the lucene bug, this should > > > work again. > > > > > > > > In the meantime I had to move a few sources from ooxml to main > > > (crypto.agile) and we need to find a way, e.g. using multi release > > > classes, for > > > our WorkbookFactory, as with JIgsaw the poi main module can't reflect into > > > ooxml anymore. This was also a reason for moving the agile crypto into poi > > > main and removing the use XmlBeans schemas there. > > > > > > > > Best wishes, > > > > Andi > > > > > > > > > > > > On 07.07.20 18:00, Dominik Stadler wrote: > > > >> Hi, > > > >> > > > >> Sorry for replying late here, not sure if you already did these > > > >> changes, > > > >> but the code in CleanerUtil tries to handle this gracefully with not > > > >> cleaning on JDKs which do not support this. > > > >> > > > >> There should be no compile-time dependency on any unsafe object, > during > > > >> runtime we try to get a cleaner and simply not unmap buffers cleanly if > > not > > > >> possible for some reason. > > > >> > > > >> Which new problem do you see with this approach when using the JDK > > > module > > > >> system? > > > >> > > > >> Thanks... Dominik. > > > >> > > > >> On Mon, Jun 29, 2020 at 10:36 PM Andreas Beeker > > > <[email protected]> > > > >> wrote: > > > >> > > > >>> Hi *, > > > >>> > > > >>> I'm facing the same problem as described in > > > >>> https://issues.apache.org/jira/browse/LUCENE-6989 > > > >>> > > > >>> Is it ok for you, if I get our build more or less to run with the > > > >>> module > > > >>> path (instead of the classpath) when running in a JDK 9+ and later > > > >>> try to > > > >>> fix the above Cleaner problem? > > > >>> > > > >>> I simply would like to focus on one issue now and as we a have > > > >>> multi-release jars after my commit, a JDK dependent solution shouldn't > > be > > > a > > > >>> problem anymore. > > > >>> > > > >>> Andi > > > >>> > > > >>> > > > >>> > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: [email protected] > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
