Hi, > I currently try to understand, how to call the Cleaner in Java 14 (or 9+) > without > adding the --add-opens JVM options.
Yeah, your code won't work correctly with Java 9 at all. You may fix it with some opens, but still types of internal calsses changed, so its just risky (everything is subject to change). > As you've worked on this in LUCENE-6989, you might have a few hints for me. I think you can more or less copy the code from Lucene (use the branch_8x version, as Master requires Java 11, so has no Java 8 code anymore, see here: https://tinyurl.com/y47euqfg). The aproach in Lucene does NOT use Reflection at all, it works with Method Handles. The important thing is: Method handles are linked earyl, once you have built the final method handle to invoke the cleaner, you can call it without requesting any additional right (security manager). In addition you know beforehand if it works at all (it cannot throw extra reflective exceptions). The main change in Java 9 is (and this is what's officially "supported" by OpenJDK developer): They added a new method to sun.misc.Unsafe (the legacy one), Unsafe#invokeCleaner(ByteBuffer). This class is still in java.base and is open to public (if you know how to get the singleton) for the time being. To get the singleton, you need reflection and the code must allow to do setAccessible on the getter, but once you have it, it's useable. To unmap a Bytebuffer, Lucene creates a MethodHandle with some signature like "unmap(ByteBuffer b)" complete pre-configued on startup of class depending on Java version: - In Java 9 and above it uses reflection to get the Unsafe instance (this requires security manager to allow it). Then it looks up the method "invokeCleaner(ByteBuffer) and binds it to the unsafe singleton, the final methodhandle is casted to me "void unmap(ByteBuffer)" - In Java 8 and before it uses more or less the old approach by checking the private method to get the Cleaner instance. Finally it calls cleaner.clean(). This can also be composed to a MethodHandle with exactly same signature (using the famous MethodHandle transformation and bindings, introducing some null checks). The result is also a methodhandle with signature "void unmap(ByteBuffer)". Once all this is done, the methodhandle with platform independent signature can be called without any exception handling from any code, so be sure to keep it safe in private final fields fully internal to your implementaion (otherwise it's a security issue). > I've checked the Lucene implementation, but that code is similar to POIs > current implementation. [1] > As I don't see the Runable interface, I might look at the wrong branch. That won't work. It's the same approach like the old one, just with other class types. You cant work around the internal Cleaner interface. This is why sun.misc.Unsafe#invokeCleaner() was added. > Any ideas? See above. I'd copy the code from Lucene: It's early binding and failsafe, once you got the MethodHandle. The approach should work with Java 7+. In Java 7 there is one additional helper method needed for the methodHandle regarding the null check! Java 8 is here: https://tinyurl.com/y47euqfg Hack for compatibility with Java 7 is here: https://tinyurl.com/y4drev3k (this adds a "compatibility method" to replace missing "Objects#nonNull(Object)", but it's identical otherwise; we added this to make Lucene 5.5 still compatible with Java 9, long after support ended, because customers need this). Uwe > Best wishes, > Andi > > > [1] > https://github.com/apache/lucene- > solr/blob/master/lucene/core/src/java/org/apache/lucene/store/MMapDirecto > ry.java#L338 > vs. > https://svn.apache.org/viewvc/poi/trunk/src/java/org/apache/poi/poifs/nio/Cle > anerUtil.java?view=markup#l91 > > On 09.07.20 00:34, Andreas Beeker wrote: > > Hi Dominik, > > > > the goal is to have no --add-opens or similar jvm arguments. In this case we > get the following exception: > > > > [junit] java.lang.reflect.InaccessibleObjectException: Unable to make > > public > jdk.internal.ref.Cleaner java.nio.DirectByteBuffer.cleaner() accessible: > module > java.base does not "opens java.nio" to module org.apache.poi.poi > > [junit] at > java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleO > bject.java:349) > > [junit] at > java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleO > bject.java:289) > > [junit] at > java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:196) > > [junit] at > java.base/java.lang.reflect.Method.setAccessible(Method.java:190) > > [junit] at > org.apache.poi.poi/org.apache.poi.poifs.nio.CleanerUtil.unmapHackImpl(Cleane > rUtil.java:116) > > [junit] at > java.base/java.security.AccessController.doPrivileged(AccessController.java:312 > ) > > [junit] at > org.apache.poi.poi/org.apache.poi.poifs.nio.CleanerUtil.<clinit>(CleanerUtil.jav > a:77) > > [junit] at > org.apache.poi.poi/org.apache.poi.poifs.nio.FileBackedDataSource.unmap(FileB > ackedDataSource.java:189) > > [junit] at > org.apache.poi.poi/org.apache.poi.poifs.nio.FileBackedDataSource.lambda$clos > e$0(FileBackedDataSource.java:162) > > [junit] at > java.base/java.util.IdentityHashMap.forEach(IdentityHashMap.java:1356) > > [junit] at > org.apache.poi.poi/org.apache.poi.poifs.nio.FileBackedDataSource.close(FileBac > kedDataSource.java:162) > > [junit] at > org.apache.poi.poi/org.apache.poi.poifs.filesystem.POIFSFileSystem.close(POIFS > FileSystem.java:764) > > [junit] at > org.apache.poi.poi/org.apache.poi.hpsf.basic.TestWrite.inPlacePOIFSWrite(Test > Write.java:539) > > > > If we use the Runnable aproach mentioned in the lucene bug, this should > work again. > > > > In the meantime I had to move a few sources from ooxml to main > (crypto.agile) and we need to find a way, e.g. using multi release classes, > for > our WorkbookFactory, as with JIgsaw the poi main module can't reflect into > ooxml anymore. This was also a reason for moving the agile crypto into poi > main and removing the use XmlBeans schemas there. > > > > Best wishes, > > Andi > > > > > > On 07.07.20 18:00, Dominik Stadler wrote: > >> Hi, > >> > >> Sorry for replying late here, not sure if you already did these changes, > >> but the code in CleanerUtil tries to handle this gracefully with not > >> cleaning on JDKs which do not support this. > >> > >> There should be no compile-time dependency on any unsafe object, during > >> runtime we try to get a cleaner and simply not unmap buffers cleanly if not > >> possible for some reason. > >> > >> Which new problem do you see with this approach when using the JDK > module > >> system? > >> > >> Thanks... Dominik. > >> > >> On Mon, Jun 29, 2020 at 10:36 PM Andreas Beeker > <[email protected]> > >> wrote: > >> > >>> Hi *, > >>> > >>> I'm facing the same problem as described in > >>> https://issues.apache.org/jira/browse/LUCENE-6989 > >>> > >>> Is it ok for you, if I get our build more or less to run with the module > >>> path (instead of the classpath) when running in a JDK 9+ and later try to > >>> fix the above Cleaner problem? > >>> > >>> I simply would like to focus on one issue now and as we a have > >>> multi-release jars after my commit, a JDK dependent solution shouldn't be > a > >>> problem anymore. > >>> > >>> Andi > >>> > >>> > >>> > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
