Hi, my question is if it is OK for a program which otherwise runs fine to fail during class verification if running with a security manager because of module access restrictions?
As the following write-up got a little lengthy, I've also uploaded a html version for a nicer reading experience :) http://cr.openjdk.java.net/~simonis/webrevs/2017/veri_with_secman/sm.html Consider the following small, self-contained program which statically references com.sun.crypto.provider.SunJCE: import java.security.Provider; import com.sun.crypto.provider.SunJCE; public class Tricky { public static void main(String args[]) throws Exception { try { System.out.println(Tricky.class + " (" + Tricky.class.getClassLoader() + ")" + args[0]); } catch (Exception e) { Provider p = new SunJCE(); System.out.println(p + " (" + p.toString() + ")"); } } } This program easily compiles and runs with Oracle/OpenJDK 8: $ javac Tricky $ java Tricky "" class Tricky (sun.misc.Launcher$AppClassLoader@4e0e2f2a) $ java Tricky SunJCE version 1.8 (SunJCE version 1.8) The second invocation (without argument) will cause an exception (when trying to access the first element of the zero length argument array at 'args[0]') which will be caught in the exception handler where we create a new 'SunJCE' object and print its string representation to stdout. The first invocation (with an empty argument) doesn't provoke an exception and will just print the class itself together with its class loader. When we run the same (jdk8 compiled) program with jdk9, we'll see the following result: $ jdk9/java Tricky "" class Tricky (jdk.internal.loader.ClassLoaders$AppClassLoader@ba8a1dc) $ jdk9/java Tricky Exception in thread "main" java.lang.IllegalAccessError: class Tricky (in unnamed module @0x3b192d32) cannot access class com.sun.crypto.provider.SunJCE (in module java.base) because module java.base does not export com.sun.crypto.provider to unnamed module @0x3b192d32 at Tricky.main(Tricky.java:11) $ jdk9/java --add-exports java.base/com.sun.crypto.provider=ALL-UNNAMED Tricky SunJCE version 9 (SunJCE version 9) The first invocation (with an empty argument) still works because we have lazy class loading and 'SunJCE' provider is actually never used. The second invocation (without argument) obviously fails with jdk9 because 'java.base' doesn't export the package com.sun.crypto.provider anymore. This can be fixed by adding '--add-exports java.base/com.sun.crypto.provider=ALL-UNNAMED' to the command line as demonstrated in the third invocation. So far so good - everything worked as expected till now. Now let's run the same program with the default security manager: $ java -Djava.security.manager Tricky "" class Tricky (sun.misc.Launcher$AppClassLoader@4e0e2f2a) $ java -Djava.security.manager Tricky SunJCE version 1.8 (SunJCE version 1.8) The security manager doesn't change anything for jdk8! So let's try with jdk9: $ jdk9/java -Djava.security.manager Tricky "" Error: A JNI error has occurred, please check your installation and try again Exception in thread "main" java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "accessClassInPackage.com.sun.crypto.provider") at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:471) at java.base/java.security.AccessController.checkPermission(AccessController.java:894) at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:561) at java.base/java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1534) at java.base/java.lang.ClassLoader$1.run(ClassLoader.java:671) at java.base/java.lang.ClassLoader$1.run(ClassLoader.java:669) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/java.lang.ClassLoader.checkPackageAccess(ClassLoader.java:669) at java.base/java.lang.Class.getDeclaredMethods0(Native Method) at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3129) at java.base/java.lang.Class.getMethodsRecursive(Class.java:3270) at java.base/java.lang.Class.getMethod0(Class.java:3256) at java.base/java.lang.Class.getMethod(Class.java:2057) at java.base/sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:712) at java.base/sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:570) The first invocation (with an empty argument) crashes with a strange JNI error. The call stack unveils that we haven't even entered the main method of our program. Instead we've crashed in the jdk-internal launcher 'sun.launcher.LauncherHelper' during 'checkAndLoadMain()' with a 'java.security.AccessControlException' because we couldn't access the 'com.sun.crypto.provider' package. That's strange because we shouldn't need to load SunJCE provider for this invocation because of lazy class loading. A little reasoning and the right Xlog parameter unveils that the exception happens during class verification: $ jdk9/java -Djava.security.manager -Xlog:verification Tricky "" ... [1,382s][info][verification] locals: { '[Ljava/lang/String;', 'java/lang/Exception', 'com/sun/crypto/provider/SunJCE' } [1,382s][info][verification] stack: { 'java/io/PrintStream', 'java/lang/StringBuilder', 'com/sun/crypto/provider/SunJCE' } [1,382s][info][verification] offset = 77, opcode = invokevirtual [1,589s][info][verification] Verification for Tricky has exception pending java.security.AccessControlException [1,589s][info][verification] End class verification for: Tricky Error: A JNI error has occurred, please check your installation and try again Exception in thread "main" java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "accessClassInPackage.com.sun.crypto.provider") ... And indeed, switching off class verification will fix the problem: $ jdk9/java -Djava.security.manager -Xlog:verification -noverify Tricky "" class Tricky (jdk.internal.loader.ClassLoaders$AppClassLoader@1b9e1916) But switching off class verification is not something we usually want to do! So we can try to use '--add-exports java.base/com.sun.crypto.provider=ALL-UNNAMED' (although this wasn't needed without security manager): $ jdk9/java -Djava.security.manager --add-exports java.base/com.sun.crypto.provider=ALL-UNNAMED Tricky "" Error: A JNI error has occurred, please check your installation and try again Exception in thread "main" java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "accessClassInPackage.com.sun.crypto.provider") ... but the result is still the same. If you feel that '--add-exports java.base/com.sun.crypto.provider=ALL-UNNAMED' should have fixed the problem, you're not alone. That issue is tracked under JDK-8174766 [1]. It was discussed in the review [2] for JDK-8055206 [3] which introduced the problem. Currently, the only way to fix the problem is to add additional permissions through a custom security policy: my.policy --------- grant { permission java.lang.RuntimePermission "accessClassInPackage.com.sun.crypto.provider"; }; $ jdk9/java -Djava.security.manager -Djava.security.policy=my.policy Tricky "" class Tricky (jdk.internal.loader.ClassLoaders$AppClassLoader@1b9e1916) But the real question remains: why does class verification fails with a security manager and succeeds without? With the right Xlog parameters we can verify, that even without security manager, the 'SunJCE' class is successfully loaded by the verifier for the purpose of verification, and the execution of the program succeeds if we don't reference 'SunJCE' during execution: $ jdk9/java -Xlog:verification -Xlog:class+load Tricky "" ... [1,193s][info][verification] locals: { '[Ljava/lang/String;', 'java/lang/Exception', 'com/sun/crypto/provider/SunJCE' } [1,193s][info][verification] stack: { 'java/io/PrintStream', 'java/lang/StringBuilder', 'com/sun/crypto/provider/SunJCE' } [1,193s][info][verification] offset = 77, opcode = invokevirtual [1,196s][info][class,load ] java.security.Provider source: jrt:/java.base [1,197s][info][class,load ] com.sun.crypto.provider.SunJCE source: jrt:/java.base ... [1,198s][info][verification] End class verification for: Tricky ... class Tricky (jdk.internal.loader.ClassLoaders$AppClassLoader@ba8a1dc) So how does the presence of a security manager changes the class verification process? With a security manager, there are two additional 'accessClassInPackage' checks during class loading. If we recall the verifier log: [1,193s][info][verification] locals: { '[Ljava/lang/String;', 'java/lang/Exception', 'com/sun/crypto/provider/SunJCE' } [1,193s][info][verification] stack: { 'java/io/PrintStream', 'java/lang/StringBuilder', 'com/sun/crypto/provider/SunJCE' } [1,193s][info][verification] offset = 77, opcode = invokevirtual [1,196s][info][class,load ] java.security.Provider source: jrt:/java.base [1,197s][info][class,load ] com.sun.crypto.provider.SunJCE source: jrt:/java.base we see that the verifier has to prove that 'com/sun/crypto/provider/SunJCE' (which is on top of the stack) can be assigned (i.e. is assign-compatible) to 'java.security.Provider' and it is therefor safe to call 'Provider::toString()' on it. Therefor, in order to load the class 'com.sun.crypto.provider.SunJCE', the verifier calls 'SystemDictionary::resolve_or_fail()' which in the end calls 'ClassLoader.loadClass(_class_name)' on the class loader of the class under verification. If that class is a user class, this will call 'jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass()' in Java 9 because that's the actual application class loader (aka system class loader) and ClassLoaders$AppClassLoader.loadClass() will perform the following check if the VM runs with a security manager: @Override protected Class loadClass(String cn, boolean resolve) throws ClassNotFoundException { SecurityManager sm = System.getSecurityManager(); if (sm != null) { int i = cn.lastIndexOf('.'); if (i != -1) { sm.checkPackageAccess(cn.substring(0, i)); } } return super.loadClass(cn, resolve); } Here's the corresponding VM stack trace: j java.lang.SecurityManager.checkPackageAccess(Ljava/lang/String;)V+46 java.base@9-internal j jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Ljava/lang/String;Z)Ljava/lang/Class;+8 java.base@9-internal j java.lang.ClassLoader.loadClass(Ljava/lang/String;)Ljava/lang/Class;+3 java.base@9-internal v ~StubRoutines::call_stub V [libjvm.so+0xc6c75d] JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, Thread*)+0x6a5 V [libjvm.so+0x1055bab] os::os_exception_wrapper(void (*)(JavaValue*, methodHandle const&, JavaCallArguments*, Thread*), JavaValue*, methodHandle const&, JavaCallArguments*, Thread*)+0x41 V [libjvm.so+0xc6c0a2] JavaCalls::call(JavaValue*, methodHandle const&, JavaCallArguments*, Thread*)+0xaa V [libjvm.so+0xc6b1b3] JavaCalls::call_virtual(JavaValue*, KlassHandle, Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x1f1 V [libjvm.so+0xc6b3f0] JavaCalls::call_virtual(JavaValue*, Handle, KlassHandle, Symbol*, Symbol*, Handle, Thread*)+0xd2 V [libjvm.so+0x1201c56] SystemDictionary::load_instance_class(Symbol*, Handle, Thread*)+0x93c V [libjvm.so+0x11feda0] SystemDictionary::resolve_instance_class_or_null(Symbol*, Handle, Handle, Thread*)+0x8c0 V [libjvm.so+0x11fd298] SystemDictionary::resolve_or_null(Symbol*, Handle, Handle, Thread*)+0x262 V [libjvm.so+0x11fcca3] SystemDictionary::resolve_or_fail(Symbol*, Handle, Handle, bool, Thread*)+0x45 V [libjvm.so+0x1286586] VerificationType::resolve_and_check_assignability(instanceKlassHandle, Symbol*, Symbol*, bool, bool, bool, Thread*)+0x258 V [libjvm.so+0x128682e] VerificationType::is_reference_assignable_from(VerificationType const&, ClassVerifier*, bool, Thread*) const+0x212 V [libjvm.so+0x118fa55] VerificationType::is_assignable_from(VerificationType const&, ClassVerifier*, bool, Thread*) const+0x191 V [libjvm.so+0x129a98f] StackMapFrame::pop_stack(VerificationType, Thread*)+0x81 V [libjvm.so+0x12978f3] ClassVerifier::verify_invoke_instructions(RawBytecodeStream*, unsigned int, StackMapFrame*, bool, bool*, VerificationType, constantPoolHandle const&, StackMapTable*, Thread*)+0x1249 V [libjvm.so+0x129096b] ClassVerifier::verify_method(methodHandle const&, Thread*)+0x6b71 V [libjvm.so+0x1289d1a] ClassVerifier::verify_class(Thread*)+0x12a V [libjvm.so+0x1287c9e] Verifier::verify(instanceKlassHandle, Verifier::Mode, bool, Thread*)+0x2c0 V [libjvm.so+0xc2b9b0] InstanceKlass::verify_code(instanceKlassHandle, bool, Thread*)+0x7c V [libjvm.so+0xc2c228] InstanceKlass::link_class_impl(instanceKlassHandle, bool, Thread*)+0x55e V [libjvm.so+0xc2bb3d] InstanceKlass::link_class(Thread*)+0xdf V [libjvm.so+0xcf5f80] get_class_declared_methods_helper(JNIEnv_*, _jclass*, unsigned char, bool, Klass*, Thread*)+0x155 V [libjvm.so+0xcf6570] JVM_GetClassDeclaredMethods+0x1d9 j java.lang.Class.getDeclaredMethods0(Z)[Ljava/lang/reflect/Method;+0 java.base@9-internal j java.lang.Class.privateGetDeclaredMethods(Z)[Ljava/lang/reflect/Method;+34 java.base@9-internal j java.lang.Class.getMethodsRecursive(Ljava/lang/String;[Ljava/lang/Class;Z)Ljava/lang/PublicMethods$MethodList;+2 java.base@9-internal j java.lang.Class.getMethod0(Ljava/lang/String;[Ljava/lang/Class;)Ljava/lang/reflect/Method;+14 java.base@9-internal j java.lang.Class.getMethod(Ljava/lang/String;[Ljava/lang/Class;)Ljava/lang/reflect/Method;+26 java.base@9-internal j sun.launcher.LauncherHelper.validateMainClass(Ljava/lang/Class;)V+12 java.base@9-internal j sun.launcher.LauncherHelper.checkAndLoadMain(ZILjava/lang/String;)Ljava/lang/Class;+54 java.base@9-internal This check succeeds, because class-loading is triggered from sun.launcher.LauncherHelper which is in the 'java.base' package, just like the 'com.sun.crypto.provider.SunJCE' package. But when running with a security manager, there's also a second check, which is performed in the VM by 'SystemDictionary::resolve_instance_class_or_null()', right before it returns the loaded class, by calling 'SystemDictionary::validate_protection_domain()'. This method in turn calls 'java.lang.ClassLoader.checkPackageAccess()' to validate the package access: // Invoked by the VM after loading class with this loader. private void checkPackageAccess(Class cls, ProtectionDomain pd) { final SecurityManager sm = System.getSecurityManager(); if (sm != null) { ... final String name = cls.getName(); final int i = name.lastIndexOf('.'); if (i != -1) { AccessController.doPrivileged(new PrivilegedAction<>() { public Void run() { sm.checkPackageAccess(name.substring(0, i)); return null; } }, new AccessControlContext(new ProtectionDomain[] {pd})); } } } This check is done with the protection domain of the initial class under verification ('Tricky' in our case) which is from the unnamed module and doesn't have the rights to access the unexported classes from java.base. Therefor the check fails the second time. Here's the corresponding VM stack trace: j java.lang.ClassLoader.checkPackageAccess(Ljava/lang/Class;Ljava/security/ProtectionDomain;)V+106 java.base@9-internal v ~StubRoutines::call_stub V [libjvm.so+0xc6c75d] JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, Thread*)+0x6a5 V [libjvm.so+0x1055bab] os::os_exception_wrapper(void (*)(JavaValue*, methodHandle const&, JavaCallArguments*, Thread*), JavaValue*, methodHandle const&, JavaCallArguments*, Thread*)+0x41 V [libjvm.so+0xc6c0a2] JavaCalls::call(JavaValue*, methodHandle const&, JavaCallArguments*, Thread*)+0xaa V [libjvm.so+0xc6b6a9] JavaCalls::call_special(JavaValue*, KlassHandle, Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x173 V [libjvm.so+0xc6ba04] JavaCalls::call_special(JavaValue*, Handle, KlassHandle, Symbol*, Symbol*, Handle, Handle, Thread*)+0xf6 V [libjvm.so+0x11fdcb0] SystemDictionary::validate_protection_domain(instanceKlassHandle, Handle, Handle, Thread*)+0x22e V [libjvm.so+0x11ff40a] SystemDictionary::resolve_instance_class_or_null(Symbol*, Handle, Handle, Thread*)+0xf2a V [libjvm.so+0x11fd298] SystemDictionary::resolve_or_null(Symbol*, Handle, Handle, Thread*)+0x262 V [libjvm.so+0x11fcca3] SystemDictionary::resolve_or_fail(Symbol*, Handle, Handle, bool, Thread*)+0x45 V [libjvm.so+0x1286586] VerificationType::resolve_and_check_assignability(instanceKlassHandle, Symbol*, Symbol*, bool, bool, bool, Thread*)+0x258 V [libjvm.so+0x128682e] VerificationType::is_reference_assignable_from(VerificationType const&, ClassVerifier*, bool, Thread*) const+0x212 V [libjvm.so+0x118fa55] VerificationType::is_assignable_from(VerificationType const&, ClassVerifier*, bool, Thread*) const+0x191 V [libjvm.so+0x129a98f] StackMapFrame::pop_stack(VerificationType, Thread*)+0x81 V [libjvm.so+0x12978f3] ClassVerifier::verify_invoke_instructions(RawBytecodeStream*, unsigned int, StackMapFrame*, bool, bool*, VerificationType, constantPoolHandle const&, StackMapTable*, Thread*)+0x1249 V [libjvm.so+0x129096b] ClassVerifier::verify_method(methodHandle const&, Thread*)+0x6b71 V [libjvm.so+0x1289d1a] ClassVerifier::verify_class(Thread*)+0x12a V [libjvm.so+0x1287c9e] Verifier::verify(instanceKlassHandle, Verifier::Mode, bool, Thread*)+0x2c0 V [libjvm.so+0xc2b9b0] InstanceKlass::verify_code(instanceKlassHandle, bool, Thread*)+0x7c V [libjvm.so+0xc2c228] InstanceKlass::link_class_impl(instanceKlassHandle, bool, Thread*)+0x55e V [libjvm.so+0xc2bb3d] InstanceKlass::link_class(Thread*)+0xdf V [libjvm.so+0xcf5f80] get_class_declared_methods_helper(JNIEnv_*, _jclass*, unsigned char, bool, Klass*, Thread*)+0x155 V [libjvm.so+0xcf6570] JVM_GetClassDeclaredMethods+0x1d9 j java.lang.Class.getDeclaredMethods0(Z)[Ljava/lang/reflect/Method;+0 java.base@9-internal j java.lang.Class.privateGetDeclaredMethods(Z)[Ljava/lang/reflect/Method;+34 java.base@9-internal j java.lang.Class.getMethodsRecursive(Ljava/lang/String;[Ljava/lang/Class;Z)Ljava/lang/PublicMethods$MethodList;+2 java.base@9-internal j java.lang.Class.getMethod0(Ljava/lang/String;[Ljava/lang/Class;)Ljava/lang/reflect/Method;+14 java.base@9-internal j java.lang.Class.getMethod(Ljava/lang/String;[Ljava/lang/Class;)Ljava/lang/reflect/Method;+26 java.base@9-internal j sun.launcher.LauncherHelper.validateMainClass(Ljava/lang/Class;)V+12 java.base@9-internal j sun.launcher.LauncherHelper.checkAndLoadMain(ZILjava/lang/String;)Ljava/lang/Class;+54 java.base@9-internal It is clear that the verifier sometimes has to load classes in order to accomplish its duty, even if these classes won't be used later on, at run-time. But the question remains if it is OK for an application to fail just because the verifier is unable to load classes required for verification because of security manager restrictions or if the verifier should run with higher privileges which allow such accesses? If the answer to this question is "Yes" (i.e. it's OK to fail), the consequence of running with a security manager will be that '--add-exports/--add-opens/--illegal-access=permit' may be not sharp enough knifes to achieve Java 8 backward compatibility. We may also have to grant some additional security permissions to our application classes. If the answer will be "No" (i.e. it's not OK to fail in the verifier) we may have to fix the VM to elevate the permissions used to load classes from within the verifier. Any comments? Thank you and best regards, Volker [1] https://bugs.openjdk.java.net/browse/JDK-8174766 [2] http://mail.openjdk.java.net/pipermail/security-dev/2017-January/thread.html#15416 [3] https://bugs.openjdk.java.net/browse/JDK-8055206
