Re: JEP 411 Headaches: Instrumenting private methods in the JDK for authorization checkpoints.
Hi Alan, My apologies, finalizer removal, that's a positive step, I thought it was motivated by "the code is trusted so we don't need to worry about finalizer attacks". Also apologies for the long email, but it seems appropriate for information handover. Atomic construction guarantee; if invariant checks during construction throw an exception, prior to calling Object's zero arg constructor, an object instance will not be created, creation of the object is atomic, either it succeeds with all invariants satisfied, or an object is not created, such that it cannot be created in a broken state where invariants aren't satisfied. We apply this constructor safety guarantee to deserialization of data, it is a requirement of the code performing deserialzation to check invariants and we require that users are authenticated, this is to avoid parsing untrusted data, while still validating user data. We enforce it using SM infrastructure. I understand JEP 411 is a business decision, there wasn't enough adoption, following the fallout of applets, businesses and users running afoul of untrusted code, causing ongoing pain (publicly). The remaining attempts of JEP 411 to explain why POLP is a technical failure only apply to the default implementation and are incorrect when applied to other implementations, it's a commercial failure, as suggested by low rates of adoption in JEP 411, but that's due to a lack of investment in tooling, however I suspect OpenJDK has underestimated adoption, although probably not by a big margin, but I suspect it will be more painful than OpenJDK anticipates. I have a perfectly good working reliable publicly available example (for years) contrary to JEP 411's technical claims. OpenJDK's decision has been made, and those affected must also assess and make their own decisions, the following only serves to share my thoughts and insights, no need to read further if not of interest. Our Java programs are going into care and maintenance as we assess suitable replacement development platforms. <-> Applets relied on SM (perhaps SM only exists due to their success), applets themselves weren't the cause of their own demise, for that we have Java Serialization to thank, otherwise applets were a commercial success, and had they remained so, then SM would have also remained, it appears to be inexorably tied to the fate of applets now. Serialization needed an atomic replacement before 2008, when it was becoming obvious that Java serialization was insecure. OpenJDK could still fix Java serialization without using white list filters (ironically white listing is a complication of SM, which reduced adoption, it's likely the same will occur with Serialization white lists, if tooling isn't provided), by removing the ability to serialize circular object graphs, or disabling it by default. We had circular object graphs in JGDMS (which heavily utilised Java serialization), but we refactored these out after implementing atomic de-serialization, we did this in a way that didn't require breaking the serial form compatibility of existing classes (unless they contained circular references). This keeps serial data invariant validation code with the object implementation, rather than as a separate white list (and it is more powerful and less complex than white listing), reducing complexity and maintenance, and because failure is atomic an attacker cannot formulate a gadget chain, type safety is also read ahead and checked prior to de-serialization of data. The development of atomic serialization started with atomic deserialization which was completed a few years ago, atomic serialization was under current development, with new explicit public API methods for used for serialization, to avoid any issues with reflection and module access controls, we were still using Java serialization to serialize, but an alternative AtomicObjectInputStream to de-serialize. SM Performance isn't an issue, my policy implementation is high scaling and has no hotspots, neither is deployment, we have tools to generate policy files (more than one) and have been doing so for many years (the first tool was written by Sun Microsystems circa 2004 I think, it still required policy file editing, but listed permissions required), the second tool was written 8 years ago approx. Our use cases have passed the tests of time. I don't believe people hand author policy files in the age of computing: I've seen examples of policy generation tools by other authors on GitHub. Sure some developers might grant AllPermission, to get something running, or for tests, but I haven't seen anyone serious about security in production that does. I don't use the built in policy provider (it has a blocking permission cache that negatively impacts performance), my policy implementation doesn't have a cache and it's many magnitudes faster and high scaling thanks to shared
Re: JEP 411 Headaches: Instrumenting private methods in the JDK for authorization checkpoints.
On 23/07/2021 23:33, Peter Firmstone wrote: I think it's worth noting that there isn't a way to securely run code with malicious intent now, so I'm surprised that at this late stage you were still providing support for sand boxing (whack a mole). It's just for us many assumptions have been made on a Java platform with SM, using POLP (not sandboxing) as this was one of the foundational principles of secure coding guidelines (just like following concurrency best practice, were were following security best practice). Sandboxing is an all or nothing approach, if you had a trusted applet that was signed, it had AllPermission, if you had an unsigned applet, then it had no permissions. Sandboxing was one of the use cases for SM, when combined with ClassLoader visibility, but we never realized that OpenJDK developers meant sandboxing == authorization access controls. When you remove that pillar, everything it's supporting collapses, not just sand boxing, so when you say you are removing support for sandboxing, we say, good idea, but we didn't realize you were saying you were removing support for all authorization access controls. Reduced and revised authorization and access control would have been acceptable, as tightening reflection visibility using a different form of access control removes the need for authorization based reflection access checks, but also removing atomic construction guarantee's just seems like were doing this at a rapid pace without the community understanding what you have in mind, and this may have more uses than just stopping finalizer attacks. I'm not 100% sure what you mean by "atomic construction guarantee" here. This JEP does not propose to change anything with finalization or do anything with the registration of finalizers after Object. runs. Our exchange in the previous mails was about classes (using ClassLoader as the example) that specify a SM permission check in their constructors, something that is strongly discouraged as the checks are easy to bypass. The idiom that we use in the JDK to prevent bypassing these SM permission checks with a finalizer attack is to check in a static method that returns a dummy parameter for the invokespecial. My point in the previous mail is that when the SM permission checks eventually go away then many of the uses of this idiom can do away too. That said, there is strong desire to eventually remove finalization too. Finalization was deprecated several years ago and the Java platform defines APIs that provide much more flexible and efficient ways do run cleanup actions when an object becomes unreachable. So another multi-year/multi-release effort to remove a problematic feature, just nothing to do with this JEP. As regards POLP, the focus of SM architecture when it was enhanced in Java 1.2. The JEP attempts to explain why this has been a failure. AccessController is voodoo that most developers have never encountered so anyone trying to run with SM ends up running counter to the principle of least privilege by granting all permissions. -Alan.
Re: JEP 411 Headaches: Instrumenting private methods in the JDK for authorization checkpoints.
I think it's worth noting that there isn't a way to securely run code with malicious intent now, so I'm surprised that at this late stage you were still providing support for sand boxing (whack a mole). It's just for us many assumptions have been made on a Java platform with SM, using POLP (not sandboxing) as this was one of the foundational principles of secure coding guidelines (just like following concurrency best practice, were were following security best practice). Sandboxing is an all or nothing approach, if you had a trusted applet that was signed, it had AllPermission, if you had an unsigned applet, then it had no permissions. Sandboxing was one of the use cases for SM, when combined with ClassLoader visibility, but we never realized that OpenJDK developers meant sandboxing == authorization access controls. When you remove that pillar, everything it's supporting collapses, not just sand boxing, so when you say you are removing support for sandboxing, we say, good idea, but we didn't realize you were saying you were removing support for all authorization access controls. Reduced and revised authorization and access control would have been acceptable, as tightening reflection visibility using a different form of access control removes the need for authorization based reflection access checks, but also removing atomic construction guarantee's just seems like were doing this at a rapid pace without the community understanding what you have in mind, and this may have more uses than just stopping finalizer attacks. Unfortunately when you develop a feature, you can't be sure developers won't adapt and utilize it for multiple purposes. Personally I think a better approach would have been to first reduce and simplify authorization access controls, replacing some of the functionality with different but more appropriate mechanisms. Removing authorization access control features, without replacement means our software would be insecure, there isn't an obvious way to re-secure it, without re-architecting or re-designing it from the ground up, and Java is now a moving target anyway, so we would have to wait for it to re-stabilize as it transitions from Hippy to Hipster Java (just making a point it's not the same Java). With this new understanding, we need to reconsider both the language and the platform that we'll be developing on. Clearly a language with less boilerplate is an obvious start, I don't yet know which, we will still consider the JVM, but it would be with Kafka or Clojure, but then we also need to consider whether we will be able to secure the underlying platform, or at least use it securely. Arguably we can do things now that aren't possible on other platforms, so we need to develop that capability as well, not just secure it. Regards, Peter. On 23/07/2021 9:45 pm, Alan Bateman wrote: On 23/07/2021 11:48, Peter Firmstone wrote: Perhaps the solution is to replace the entire class, instead of instrumenting one method? Compile a patched copy of the JVM, with modified class files, then replace the existing classes in the JVM with the modified classes? Kinda like maintaining a fork, but using Agents to instrument the original JVM with classes from the fork? I sure wish there was a better option, if anyone knows one, I'm all ears. JEP 411 puts the JDK on the road to dropping support for sandboxing. This means there won't be a built-in means to securely run code that has malicious intent. It means that many of the concerns for finalizer attacks go away too. In the case of the ClassLoader example in your first mail then it may be that the private static method that you want to instrument will be removed. If removed, then it should make the instrumentation a bit easier so that you can instrument the protected constructors to invokestatic your equivalent of a permission check before the invokespecial. So I think this specific case is surmountable but in general I don't think it will be tenable to patch hundreds of classes and be confident that you've got everything, esp. with a moving code base and new features. I can't tell if your "authorization layer" is for use when running with code that has malicious intent. If it is, then I don't think it will be tenable to duplicate all the deeply invasive permission checks that exist today and keep it up to date as new features and changes. When agents were muted in the early discussion on JEP 411 then the context was file and network access where several people were interested in having a means to veto access. Expanding this to have a SM equivalent be able to veto every reflective access, prevent trusted method chain and other attacks, amounts to keeping the SM forever. As regards the comments about agents having the power to instrument methods that aren't accessible to user code then that is normal. Java agents are for tools to do powerful things, they aren't intended
Re: JEP 411 Headaches: Instrumenting private methods in the JDK for authorization checkpoints.
Trusted code only, but it's intended to be run in a locked down principle of least privilege environment, while we perform static analysis and targeted review, we don't have the resources required to perform thorough trusted code audits, so have been reliant on the principle of least privilege. Hence the ability to generate policy files that we use for auditing. I utilize the constructor guarantee added in Java 6, that prevents finalizer attacks for atomic de-serialization, if invariants aren't satisfied, failure is atomic, thanks to this feature. We use the SM to ensure network connections used are secure and only have authenticated users, otherwise de-serialization and dynamic class loading is prevented. We don't need all the SM permission check features, but preventing unauthorized class loading is one of them. Network, file access, class loading, properties are a few that come to mind, I'm not worried about reflection, other recent changes are taking care of reflective access. We have a heap of our own Permission implementations as well. It appears that Java's future path is diverging from our software's needs and Java won't be a suitable platform in future. I wasn't aware that Java was heading in this direction. JEP 411 came as a big surprise, it just seemed like it was a core Java feature. I think the best course of action, will be to focus on Java 8, until that's EOL'd. I do like the improvements to TLS stateless connections and new features in later Java versions, but it looks like a mistake to try and keep up with Java, with this new knowledge. I think I'll back out some more recent changes that were intended to allow our software to run on Java 17, as these cause breakages for downstream developers and it doesn't seem to make much sense now, to make those breaking changes. I was actually looking forward to taking advantage of loom and the new vector API, but it looks like I need to be focused on what's going to be our next software platform. I've always found llvm to be interesting, so maybe I'll focus on some languages that run on that, and implement privileged constraints at a lower level. Perhaps by the time Java 8's EOL'd there will be a new platform and language that's more suitable. Thank you for your time, I do appreciate your replies and it gives me the clarity I needed. I wish it wasn't the case, but I have to accept that times are changing. Best of luck for the future my Java friends, it's been a powerful language, especially with regard to concurrency and scaling, we got great performance, all our hotspots were native methods and Java let us get close to the metal at low levels as well using bit shift operations on primitives, that are magnitudes faster than standard string operations. TLS ran so much faster with stateless session tickets too. Regards, Peter. On 23/07/2021 9:45 pm, Alan Bateman wrote: On 23/07/2021 11:48, Peter Firmstone wrote: Perhaps the solution is to replace the entire class, instead of instrumenting one method? Compile a patched copy of the JVM, with modified class files, then replace the existing classes in the JVM with the modified classes? Kinda like maintaining a fork, but using Agents to instrument the original JVM with classes from the fork? I sure wish there was a better option, if anyone knows one, I'm all ears. JEP 411 puts the JDK on the road to dropping support for sandboxing. This means there won't be a built-in means to securely run code that has malicious intent. It means that many of the concerns for finalizer attacks go away too. In the case of the ClassLoader example in your first mail then it may be that the private static method that you want to instrument will be removed. If removed, then it should make the instrumentation a bit easier so that you can instrument the protected constructors to invokestatic your equivalent of a permission check before the invokespecial. So I think this specific case is surmountable but in general I don't think it will be tenable to patch hundreds of classes and be confident that you've got everything, esp. with a moving code base and new features. I can't tell if your "authorization layer" is for use when running with code that has malicious intent. If it is, then I don't think it will be tenable to duplicate all the deeply invasive permission checks that exist today and keep it up to date as new features and changes. When agents were muted in the early discussion on JEP 411 then the context was file and network access where several people were interested in having a means to veto access. Expanding this to have a SM equivalent be able to veto every reflective access, prevent trusted method chain and other attacks, amounts to keeping the SM forever. As regards the comments about agents having the power to instrument methods that aren't accessible to user code then that is normal. Java agents are for tools to
Re: JEP 411 Headaches: Instrumenting private methods in the JDK for authorization checkpoints.
On 23/07/2021 11:48, Peter Firmstone wrote: Perhaps the solution is to replace the entire class, instead of instrumenting one method? Compile a patched copy of the JVM, with modified class files, then replace the existing classes in the JVM with the modified classes? Kinda like maintaining a fork, but using Agents to instrument the original JVM with classes from the fork? I sure wish there was a better option, if anyone knows one, I'm all ears. JEP 411 puts the JDK on the road to dropping support for sandboxing. This means there won't be a built-in means to securely run code that has malicious intent. It means that many of the concerns for finalizer attacks go away too. In the case of the ClassLoader example in your first mail then it may be that the private static method that you want to instrument will be removed. If removed, then it should make the instrumentation a bit easier so that you can instrument the protected constructors to invokestatic your equivalent of a permission check before the invokespecial. So I think this specific case is surmountable but in general I don't think it will be tenable to patch hundreds of classes and be confident that you've got everything, esp. with a moving code base and new features. I can't tell if your "authorization layer" is for use when running with code that has malicious intent. If it is, then I don't think it will be tenable to duplicate all the deeply invasive permission checks that exist today and keep it up to date as new features and changes. When agents were muted in the early discussion on JEP 411 then the context was file and network access where several people were interested in having a means to veto access. Expanding this to have a SM equivalent be able to veto every reflective access, prevent trusted method chain and other attacks, amounts to keeping the SM forever. As regards the comments about agents having the power to instrument methods that aren't accessible to user code then that is normal. Java agents are for tools to do powerful things, they aren't intended for libraries for applications to use directly. This is why agents are opt-in on the command line. Agent maintainers weld great power and must take care to never leak the Instrumentation object to applications or libraries. -Alan
Re: JEP 411 Headaches: Instrumenting private methods in the JDK for authorization checkpoints.
Perhaps the solution is to replace the entire class, instead of
instrumenting one method?
Compile a patched copy of the JVM, with modified class files, then
replace the existing classes in the JVM with the modified classes?
Kinda like maintaining a fork, but using Agents to instrument the
original JVM with classes from the fork?
I sure wish there was a better option, if anyone knows one, I'm all ears.
Regards,
Peter.
On 23/07/2021 6:36 pm, Peter Firmstone wrote:
Post JEP 411, we need to write Agents to replace the current
permission checks performed within the JVM by instrumenting it,
following advice by OpenJDK developers, however for us this goes
against all our previous development practices, no part of our
codebase accesses any JDK implementation classes or internal api's.
We also don't release anything that depends on deprecated API's.
Modules didn't break our code, neither has tightening rules around
access in JEP 403.
We have been advised that we need to instrument the JDK with Agent's
by OpenJDK.
I am now ready to write these Agents, so that I can begin testing my
new authorization layer for Java:
https://github.com/pfirmstone/HighPerformanceSecurity
As an example, we need to instrument java.lang.ClassLoader, in this
case we need to instrument the following method:
private static Void checkCreateClassLoader(String name); This check
must be performed prior to calling java.lang.Object's constructor, to
throw an exception, without making ClassLoader susceptible to a
finalizer attack.
Accessing private internal methods goes against all our current
development practices, these are at risk of breaking in future.
We cannot add methods with Agent's only change method contents.
I am requesting hooks, in the form of an annotation, such as the
following, so that OpenJDK developers know that this method will be
instrumented by Agent's and not to change it's signature.
@Instrumented
private static Void checkCreateClassLoader(String name);
If OpenJDK will provide instrumentation hooks, then this is a workable
solution for us.
Or is OpenJDK encouraging us to start accessing private methods and
have to test each Java release for breakages?
I'm wondering what the point of JEP 403 is if, our solution to JEP
411, is to start instrumenting private methods? I don't think this
is what OpenJDK developers are proposing.
Currently removing SM will allow an attacker to cause our software
running on the JVM to parse untrusted data, which previously required
an authenticated client? Permission is only granted to Principal's,
of course post JEP 411, these checks will stop working in future,
making our software vulnerable to attacks by unauthenticated users.
We're still up in the air about how to provide credentials for our TLS
and Kerberos connections, for authentication, I've created support for
obtaining the Subject from the Authorization context, in my
authorization layer software (linked above), but instrumenting private
methods in the JDK goes against all previously learned best practices.
--
Regards,
Peter Firmstone
Code snippet from java.lang.ClassLoader:
if (name != null && name.isEmpty()) {
throw new IllegalArgumentException("name must be non-empty or null");
}
@SuppressWarnings("removal")
SecurityManager security = System.getSecurityManager();
if (security != null) {
security.checkCreateClassLoader();
}
return null;
}
private ClassLoader(Void unused, String name, ClassLoader parent) {
this.name = name;
this.parent = parent;
this.unnamedModule = new Module(this);
if (ParallelLoaders.isRegistered(this.getClass())) {
parallelLockMap = new ConcurrentHashMap<>();
assertionLock = new Object();
} else {
// no finer-grained lock; lock on the classloader instance
parallelLockMap = null;
assertionLock = this;
}
this.package2certs = new ConcurrentHashMap<>();
this.nameAndId = nameAndId(this);
}
/**
* If the defining loader has a name explicitly set then
* '' @
* If the defining loader has no name then
* @
* If it's built-in loader then omit `@` as there is only one
instance.
*/
private static String nameAndId(ClassLoader ld) {
String nid = ld.getName() != null ? "\'" + ld.getName() + "\'"
: ld.getClass().getName();
if (!(ld instanceof BuiltinClassLoader)) {
String id = Integer.toHexString(System.identityHashCode(ld));
nid = nid + " @" + id;
}
return nid;
}
/**
* Creates a new class loader of the specified name and using the
* specified parent class loader for delegation.
*
* @apiNote If the parent is specified as {@codenull} (for the
* bootstrap class loader) then there is no guarantee that all platform
* classes are visible.
*
* @param name class loader name; or {@codenull} if not named
* @param parent the parent class loader
*
* @throws IllegalArgumentException if the given name is empty.
*
* @throws SecurityException
* If a security manager exists and its
* {@link
JEP 411 Headaches: Instrumenting private methods in the JDK for authorization checkpoints.
Post JEP 411, we need to write Agents to replace the current permission
checks performed within the JVM by instrumenting it, following advice by
OpenJDK developers, however for us this goes against all our previous
development practices, no part of our codebase accesses any JDK
implementation classes or internal api's. We also don't release anything
that depends on deprecated API's. Modules didn't break our code, neither
has tightening rules around access in JEP 403.
We have been advised that we need to instrument the JDK with Agent's by
OpenJDK.
I am now ready to write these Agents, so that I can begin testing my new
authorization layer for Java:
https://github.com/pfirmstone/HighPerformanceSecurity
As an example, we need to instrument java.lang.ClassLoader, in this case
we need to instrument the following method:
private static Void checkCreateClassLoader(String name); This check
must be performed prior to calling java.lang.Object's constructor, to
throw an exception, without making ClassLoader susceptible to a
finalizer attack.
Accessing private internal methods goes against all our current
development practices, these are at risk of breaking in future.
We cannot add methods with Agent's only change method contents.
I am requesting hooks, in the form of an annotation, such as the
following, so that OpenJDK developers know that this method will be
instrumented by Agent's and not to change it's signature.
@Instrumented
private static Void checkCreateClassLoader(String name);
If OpenJDK will provide instrumentation hooks, then this is a workable
solution for us.
Or is OpenJDK encouraging us to start accessing private methods and have
to test each Java release for breakages?
I'm wondering what the point of JEP 403 is if, our solution to JEP 411,
is to start instrumenting private methods? I don't think this is what
OpenJDK developers are proposing.
Currently removing SM will allow an attacker to cause our software
running on the JVM to parse untrusted data, which previously required an
authenticated client? Permission is only granted to Principal's, of
course post JEP 411, these checks will stop working in future, making
our software vulnerable to attacks by unauthenticated users.
We're still up in the air about how to provide credentials for our TLS
and Kerberos connections, for authentication, I've created support for
obtaining the Subject from the Authorization context, in my
authorization layer software (linked above), but instrumenting private
methods in the JDK goes against all previously learned best practices.
--
Regards,
Peter Firmstone
Code snippet from java.lang.ClassLoader:
if (name != null && name.isEmpty()) {
throw new IllegalArgumentException("name must be non-empty or null");
}
@SuppressWarnings("removal")
SecurityManager security = System.getSecurityManager();
if (security != null) {
security.checkCreateClassLoader();
}
return null;
}
private ClassLoader(Void unused, String name, ClassLoader parent) {
this.name = name;
this.parent = parent;
this.unnamedModule = new Module(this);
if (ParallelLoaders.isRegistered(this.getClass())) {
parallelLockMap = new ConcurrentHashMap<>();
assertionLock = new Object();
} else {
// no finer-grained lock; lock on the classloader instance
parallelLockMap = null;
assertionLock = this;
}
this.package2certs = new ConcurrentHashMap<>();
this.nameAndId = nameAndId(this);
}
/**
* If the defining loader has a name explicitly set then
* '' @
* If the defining loader has no name then
* @
* If it's built-in loader then omit `@` as there is only one instance.
*/
private static String nameAndId(ClassLoader ld) {
String nid = ld.getName() != null ? "\'" + ld.getName() + "\'"
: ld.getClass().getName();
if (!(ld instanceof BuiltinClassLoader)) {
String id = Integer.toHexString(System.identityHashCode(ld));
nid = nid + " @" + id;
}
return nid;
}
/**
* Creates a new class loader of the specified name and using the
* specified parent class loader for delegation.
*
* @apiNote If the parent is specified as {@codenull} (for the
* bootstrap class loader) then there is no guarantee that all platform
* classes are visible.
*
* @param name class loader name; or {@codenull} if not named
* @param parent the parent class loader
*
* @throws IllegalArgumentException if the given name is empty.
*
* @throws SecurityException
* If a security manager exists and its
* {@link SecurityManager#checkCreateClassLoader()}
* method doesn't allow creation of a new class loader.
*
* @since 9
*/
protected ClassLoader(String name, ClassLoader parent) {
this(checkCreateClassLoader(name), name, parent);
}
private static Void checkCreateClassLoader(String name) {
