Re: JEP 411: Deprecation with removal would break most existing Java libraries
On 16/06/2021 11:18 pm, David Lloyd wrote: There are already issues with Permission implementations, take for example SocketPermission, it consults DNS and it isn't possible to enter a range of IP addresses (such as the local subnet, and a list of public IP addresses), for now, every single IP address must be entered and this isn't practical. The proposed API would allow us to re-implement SocketPermission functionality, as well as other Permission implementations. Sure, this would be nice to clean up. What the above example enhances: * Generation of policy files during integration testing. * Specifying Properties, to replace local information contained in URL and file paths, for later policy expansion. * In this specific case, I could substitute an IP address with a property that specified an allowed subnet mask, this would automatically expand the SocketPermission to the local subnet, and it would shrink the size of the generated policy file by eliminating other SocketPermission grants to IP addresses on the subnet. -- Regards, Peter Firmstone Zeus Project Services Pty Ltd.
Re: JEP 411: Deprecation with removal would break most existing Java libraries
On 18/06/2021 1:18 pm, Peter Firmstone wrote:
On 16/06/2021 11:18 pm, David Lloyd wrote:
On Mon, Jun 14, 2021 at 6:47 PM Peter Firmstone
wrote:
Permission references can be replaced with Guard references (which
Permissions are instances of).
I guess you've got something fairly complex in mind, could you give
some practical examples of how this would work?
The same service provider mechanism encryption uses. So
implementation may utilize authorization access check points without
any dependencies on current SecurityManager, Policy or Permission API's.
Eg GuardFactorySpi
It's completely up to the implementation to determine how to manage.
The Permission implementations of Guard::check call SecurityManager, so
checks will continue working as expected, but it allows us to intercept
them and do something different.
What do you envision these checks looking like? Where would the JDK
find these Guard instances?
GuardFactory socketGuardFactory = GuardFactory.getInstance("SOCKET");
Guard localhostConnectAccept = socketGuardFactory.orders("localhost",
"connect,accept");
// Permission check
localHostConnectAccept.check();
GuardFactory runtimeGuardFactory = GuardFactory.getInstance("RUNTIME");
Guard createClassLoader =
runtimeGuardFactory.orders("createClassLoader", null);
// Permission check
createClassLoader.check();
--
Regards,
Peter Firmstone
0498 286 363
Zeus Project Services Pty Ltd.
Re: JEP 411: Deprecation with removal would break most existing Java libraries
On 16/06/2021 11:18 pm, David Lloyd wrote:
On Mon, Jun 14, 2021 at 6:47 PM Peter Firmstone
wrote:
Permission references can be replaced with Guard references (which
Permissions are instances of).
I guess you've got something fairly complex in mind, could you give
some practical examples of how this would work?
The same service provider mechanism encryption uses. So implementation
may utilize authorization access check points without any dependencies
on current SecurityManager, Policy or Permission API's.
It's completely up to the implementation to determine how to manage.
The Permission implementations of Guard::check call SecurityManager, so
checks will continue working as expected, but it allows us to intercept
them and do something different.
What do you envision these checks looking like? Where would the JDK
find these Guard instances?
GuardFactory socketGuardFactory = GuardFactory.getInstance("SOCKET");
Guard localhostConnectAccept = socketGuardFactory.orders("localhost",
"connect,accept");
// Permission check
localHostConnectAccept.check();
GuardFactory runtimeGuardFactory = GuardFactory.getInstance("RUNTIME");
Guard createClassLoader =
runtimeGuardFactory.orders("createClassLoader", null);
// Permission check
createClassLoader.check();
--
Regards,
Peter Firmstone
0498 286 363
Zeus Project Services Pty Ltd.
Re: JEP 411: Deprecation with removal would break most existing Java libraries
On Mon, Jun 14, 2021 at 6:47 PM Peter Firmstone wrote: > SecurityManager depends on Permission, currently there are Permission > checks throughout the JVM, however Permission implementation classes > will be removed, although the Permission class itself won't be. This is incorrect AFAICT. The relevant JEP text is: > Permission and subclasses — Other significant classes, such as > ProtectionDomain, depend on Permission. Many of the subclasses of Permission, > however, are specific to use cases which will likely no longer be relevant > after the Security Manager is removed. The maintainers of these subclasses > can deprecate and remove them separately, after evaluating the compatibility > risk. > Permission references can be replaced with Guard references (which > Permissions are instances of). I guess you've got something fairly complex in mind, could you give some practical examples of how this would work? > The Permission implementations of Guard::check call SecurityManager, so > checks will continue working as expected, but it allows us to intercept > them and do something different. What do you envision these checks looking like? Where would the JDK find these Guard instances? > By replacing Permission references with Guard, it allows us to implement > our own checks in these locations, and OpenJDK doesn't need to maintain > Permission instances, and or, we don't need to make use of unmaintained > Permission implementations. As I said I don't think there is an intention to remove the permission classes just yet, and I don't think that it is a fair statement to say that the permission implementations would be unmaintained. Most of those classes have not needed to be touched in many years; there's just not a lot of complexity there for the vast majority of them. > There are already issues with Permission implementations, take for > example SocketPermission, it consults DNS and it isn't possible to enter > a range of IP addresses (such as the local subnet, and a list of public > IP addresses), for now, every single IP address must be entered and this > isn't practical. The proposed API would allow us to re-implement > SocketPermission functionality, as well as other Permission implementations. Sure, this would be nice to clean up. I just don't understand the proposed mechanism. > > On Mon, Jun 14, 2021 at 12:57 AM Alan Bateman > > wrote: > >> AccessController::doPriv just runs the action. > > TBH this should have always been the case. Implementation-wise, if > > one were constructing an access control context based on stack > > walking, one would stop at points where `AccessController` is on the > > stack (which is easily determinable) to do special work on assembling > > the access control context based on the method called at that frame. > > Yes, one can do that, but these classes will also eventually be removed. Sure. This was mainly a practical observation about the current implementation. And any replacement third-party stack-based authorization system could (and should) use a similar mechanism regardless of whether these exact methods stay in the JDK for 5 or 50 more years. -- - DML • he/him
Re: JEP 411: Deprecation with removal would break most existing Java libraries
On 14/06/2021 9:34 pm, Rafael Winterhalter wrote: Why not add the property once this is the case, though? As it is now, I read the 'forRemoval' property to indicate a problem that should be instantly addressed. I too suggested and support this approach. With Java 8 being a common baseline for libraries and the version being supported until (at least) 2030, I don't see how this removal would have a low impact within the next decade, if ever. Shouldn't the property be set if the removal is within reach? To some degree, I would expect that any deprecated API could be removed once it is no longer used. As it is now, library maintainers face the choice of breaking their support for current users that are on Java 8/11 and rely on the security manager, or to remove their support to accommodate a Java release that might be many years in the future. For this reason, I'm proposing a minimal change allowing us to implement Guard::check hooks at existing check points within the JVM using the security provider mechanism that can be back-ported and supported on all LTS releases, without any new Java API's. We require authorization layer functionality, I will be implementing it, and it will be freely available under an AL2.0 license. It would be nice to keep AccessController and AccessControlContext and use a property to enable or disable the stack walk for those who don't require it, however they are now marked for removal, so I'll be looking at using wrapper classes around them, so their implementations can be replaced at a later data. Our current implementation is high scaling with minimal performance impact, however I can't make any promises regarding future performance, but hopefully it will come to be as performant as our current implementation. Regards, Peter. For my part, supporting the security manager seems to be the right choice as things stand today. Over the years, I would expect that fewer and fewer people rely on the security manager, where this balance might shift. I would hope that the 'forRemoval' property would serve as an indicator at that time to tell library maintainers that usage of the security manager has decreased so much that it is time to remove the library support, too. I see the reason for a strong signal, deprecation already is such a signal, but if you give the full blow today, it is no longer available in the future where it might be more relevant to give. Am Mo., 14. Juni 2021 um 11:26 Uhr schrieb Ron Pressler mailto:ron.press...@oracle.com>>: The JEP addresses this: > In future JDK releases, we will degrade the Security Manager APIs so that they remain in place but have limited or no functionality. ... This will allow libraries that support the Security Manager and were compiled against previous Java releases to continue to work without change or even recompilation. We expect to remove the APIs once the compatibility risk of doing so declines to an acceptable level. If your question is, when only few codebases will refer to the API and the compatibility impact is low enough for it to be removed, what if some of those few libraries still want to support versions prior to the removal, the answer is the same as with all removals (MR-JARs, or multiple artefacts). By definition, the impact of complete removal when the impact is low, would be low. — Ron > On 14 Jun 2021, at 09:38, Rafael Winterhalter mailto:rafael@gmail.com>> wrote: > > One example for a currently necessary "doPrivileged" are Java agents where > a class loading triggers agent code where the agent shares the stack with > any code that loads a class for the first time. Otherwise, Byte Buddy wraps > anything that might require privileges as privileged action to allow > setting a policy that gives Byte Buddy for example access to class loaders, > system properties or other things that the security manager currently > checks. There's many uses of the security manager throughout the library, > in the spirit of the API's invention. > > I could, of course, rip this code out of the library. But this would make > it impossible for users that choose to use the functionality for now to > update their dependency. This would certainly hinder a smooth transition as > library maintainers will always have people drag at both ends of the JDK > version range. After all, Java 8 is supported for another decade. > Multi-release jars are neither a feasible option. They are not globally > supported by all class loaders, and would require me to add a copy of an > adjusted class file for any Java version prior to the removal version or > upwards from there. I don't think that this should be addressed by tooling > if keeping deprecated skeletons of the API can so
Re: JEP 411: Deprecation with removal would break most existing Java libraries
On 15/06/2021 2:23 am, David Lloyd wrote: On Mon, Jun 14, 2021 at 2:38 AM Peter Firmstone wrote: 1. Develop authorization layer security provider services in OpenJDK, back port it to Java 8 and Java 11 (these provide most of the utilised functionality of SecurityManager, allowing developers to only implement those which they need, without enabling SecurityManager and editing policy files). 2. Remove SecurityManager, Policy and Policy provider in OpenJDK 19. The SecurityManager class itself already is *exactly* an authorization provider. I don't think it makes sense to consider removing the security manager class to replace it with something that has basically exactly the same API Logic behind this choice: SecurityManager depends on Permission, currently there are Permission checks throughout the JVM, however Permission implementation classes will be removed, although the Permission class itself won't be. Permission references can be replaced with Guard references (which Permissions are instances of). The Permission implementations of Guard::check call SecurityManager, so checks will continue working as expected, but it allows us to intercept them and do something different. By replacing Permission references with Guard, it allows us to implement our own checks in these locations, and OpenJDK doesn't need to maintain Permission instances, and or, we don't need to make use of unmaintained Permission implementations. There are already issues with Permission implementations, take for example SocketPermission, it consults DNS and it isn't possible to enter a range of IP addresses (such as the local subnet, and a list of public IP addresses), for now, every single IP address must be entered and this isn't practical. The proposed API would allow us to re-implement SocketPermission functionality, as well as other Permission implementations. This proposal also allows every existing component of the SM architecture to be removed, while retaining the most important component, the checks themselves, such that you or I or anyone else for that matter can re-implement the functionality of SM. SM and friends will be removed eventually, so now is our opportunity to get something in place that has minimal impact on OpenJDK maintenance, that will remain. (specifically, a single method for each general authorization check that can be called without constructing any new objects, if and only if the authorization provider is installed). See my other proposal where, post-"removal", SecurityManager (the class) is retained but made abstract (and sans a few methods). All of the existing code which performs authorization checks would be retained and the problem solved in essentially the way you're describing, just using existing APIs. The security manager implementation itself can implement any kind of authorization behavior whatsoever, based mainly on the Permission types (which work just fine for this purpose, and anyway are already retained by the current JEP). Policy and its supporting classes are completely unnecessary for implementing a security policy. In fact, this is the case today already. On Mon, Jun 14, 2021 at 12:57 AM Alan Bateman wrote: AccessController::doPriv just runs the action. TBH this should have always been the case. Implementation-wise, if one were constructing an access control context based on stack walking, one would stop at points where `AccessController` is on the stack (which is easily determinable) to do special work on assembling the access control context based on the method called at that frame. Yes, one can do that, but these classes will also eventually be removed. -- Regards, Peter Firmstone 0498 286 363 Zeus Project Services Pty Ltd.
Re: JEP 411: Deprecation with removal would break most existing Java libraries
On Mon, Jun 14, 2021 at 2:38 AM Peter Firmstone wrote: > 1. Develop authorization layer security provider services in OpenJDK, > back port it to Java 8 and Java 11 (these provide most of the > utilised functionality of SecurityManager, allowing developers to > only implement those which they need, without enabling > SecurityManager and editing policy files). > 2. Remove SecurityManager, Policy and Policy provider in OpenJDK 19. The SecurityManager class itself already is *exactly* an authorization provider. I don't think it makes sense to consider removing the security manager class to replace it with something that has basically exactly the same API (specifically, a single method for each general authorization check that can be called without constructing any new objects, if and only if the authorization provider is installed). See my other proposal where, post-"removal", SecurityManager (the class) is retained but made abstract (and sans a few methods). All of the existing code which performs authorization checks would be retained and the problem solved in essentially the way you're describing, just using existing APIs. The security manager implementation itself can implement any kind of authorization behavior whatsoever, based mainly on the Permission types (which work just fine for this purpose, and anyway are already retained by the current JEP). Policy and its supporting classes are completely unnecessary for implementing a security policy. In fact, this is the case today already. On Mon, Jun 14, 2021 at 12:57 AM Alan Bateman wrote: > AccessController::doPriv just runs the action. TBH this should have always been the case. Implementation-wise, if one were constructing an access control context based on stack walking, one would stop at points where `AccessController` is on the stack (which is easily determinable) to do special work on assembling the access control context based on the method called at that frame. -- - DML • he/him
Re: JEP 411: Deprecation with removal would break most existing Java libraries
On 6/14/21 7:34 AM, Rafael Winterhalter wrote: Why not add the property once this is the case, though? As it is now, I read the 'forRemoval' property to indicate a problem that should be instantly addressed. With Java 8 being a common baseline for libraries and the version being supported until (at least) 2030, I don't see how this removal would have a low impact within the next decade, if ever. Shouldn't the property be set if the removal is within reach? To some degree, I would expect that any deprecated API could be removed once it is no longer used. > As it is now, library maintainers face the choice of breaking their support for current users that are on Java 8/11 and rely on the security manager, or to remove their support to accommodate a Java release that might be many years in the future. For my part, supporting the security manager seems to be the right choice as things stand today. Over the years, I would expect that fewer and fewer people rely on the security manager, where this balance might shift. I would hope that the 'forRemoval' property would serve as an indicator at that time to tell library maintainers that usage of the security manager has decreased so much that it is time to remove the library support, too. I see the reason for a strong signal, deprecation already is such a signal, but if you give the full blow today, it is no longer available in the future where it might be more relevant to give. As Alan and others have mentioned, there is no immediate plan to remove the APIs that SM-enabled libraries are typically dependent on, i.e., AccessController.doPrivileged and System.getSecurityManager. The APIs, when degraded will behave as if the Security Manager was not enabled. It isn't clear to me why that plan would have a high compatibility impact for libraries. And in JDK 17, the Security Manager will still be fully supported. The 'forRemoval' property means that the API is earmarked for removal in a future release. This is important, as it provides a clear warning bell to those that are dependent on it that it will eventually be going away. On the other hand, if an API is only deprecated, there is no intention to remove it. Users usually can still expect the API to continue to work as specified. It isn't as strong enough of a signal that it should not be used and in my opinion would only delay the removal of the Security Manager for many more years. I also think it would be more confusing, and possibly wrong to degrade or "stub" a deprecated API (one that is not marked forRemoval) such that it is a no-op or basically no longer works as expected. --Sean Am Mo., 14. Juni 2021 um 11:26 Uhr schrieb Ron Pressler < ron.press...@oracle.com>: The JEP addresses this: In future JDK releases, we will degrade the Security Manager APIs so that they remain in place but have limited or no functionality. ... This will allow libraries that support the Security Manager and were compiled against previous Java releases to continue to work without change or even recompilation. We expect to remove the APIs once the compatibility risk of doing so declines to an acceptable level. If your question is, when only few codebases will refer to the API and the compatibility impact is low enough for it to be removed, what if some of those few libraries still want to support versions prior to the removal, the answer is the same as with all removals (MR-JARs, or multiple artefacts). By definition, the impact of complete removal when the impact is low, would be low. — Ron On 14 Jun 2021, at 09:38, Rafael Winterhalter wrote: One example for a currently necessary "doPrivileged" are Java agents where a class loading triggers agent code where the agent shares the stack with any code that loads a class for the first time. Otherwise, Byte Buddy wraps anything that might require privileges as privileged action to allow setting a policy that gives Byte Buddy for example access to class loaders, system properties or other things that the security manager currently checks. There's many uses of the security manager throughout the library, in the spirit of the API's invention. I could, of course, rip this code out of the library. But this would make it impossible for users that choose to use the functionality for now to update their dependency. This would certainly hinder a smooth transition as library maintainers will always have people drag at both ends of the JDK version range. After all, Java 8 is supported for another decade. Multi-release jars are neither a feasible option. They are not globally supported by all class loaders, and would require me to add a copy of an adjusted class file for any Java version prior to the removal version or upwards from there. I don't think that this should be addressed by tooling if keeping deprecated skeletons of the API can so easily avoid this entire problem for all libraries without the need to chase down maintainers.
Re: JEP 411: Deprecation with removal would break most existing Java libraries
Why not add the property once this is the case, though? As it is now, I read the 'forRemoval' property to indicate a problem that should be instantly addressed. With Java 8 being a common baseline for libraries and the version being supported until (at least) 2030, I don't see how this removal would have a low impact within the next decade, if ever. Shouldn't the property be set if the removal is within reach? To some degree, I would expect that any deprecated API could be removed once it is no longer used. As it is now, library maintainers face the choice of breaking their support for current users that are on Java 8/11 and rely on the security manager, or to remove their support to accommodate a Java release that might be many years in the future. For my part, supporting the security manager seems to be the right choice as things stand today. Over the years, I would expect that fewer and fewer people rely on the security manager, where this balance might shift. I would hope that the 'forRemoval' property would serve as an indicator at that time to tell library maintainers that usage of the security manager has decreased so much that it is time to remove the library support, too. I see the reason for a strong signal, deprecation already is such a signal, but if you give the full blow today, it is no longer available in the future where it might be more relevant to give. Am Mo., 14. Juni 2021 um 11:26 Uhr schrieb Ron Pressler < ron.press...@oracle.com>: > The JEP addresses this: > > > In future JDK releases, we will degrade the Security Manager APIs so > that they > remain in place but have limited or no functionality. ... This will allow > libraries > that support the Security Manager and were compiled against previous Java > releases > to continue to work without change or even recompilation. We expect to > remove the > APIs once the compatibility risk of doing so declines to an acceptable > level. > > If your question is, when only few codebases will refer to the API and the > compatibility impact is low enough for it to be removed, what if some of > those > few libraries still want to support versions prior to the removal, the > answer is > the same as with all removals (MR-JARs, or multiple artefacts). By > definition, > the impact of complete removal when the impact is low, would be low. > > — Ron > > > On 14 Jun 2021, at 09:38, Rafael Winterhalter > wrote: > > > > One example for a currently necessary "doPrivileged" are Java agents > where > > a class loading triggers agent code where the agent shares the stack with > > any code that loads a class for the first time. Otherwise, Byte Buddy > wraps > > anything that might require privileges as privileged action to allow > > setting a policy that gives Byte Buddy for example access to class > loaders, > > system properties or other things that the security manager currently > > checks. There's many uses of the security manager throughout the library, > > in the spirit of the API's invention. > > > > I could, of course, rip this code out of the library. But this would make > > it impossible for users that choose to use the functionality for now to > > update their dependency. This would certainly hinder a smooth transition > as > > library maintainers will always have people drag at both ends of the JDK > > version range. After all, Java 8 is supported for another decade. > > Multi-release jars are neither a feasible option. They are not globally > > supported by all class loaders, and would require me to add a copy of an > > adjusted class file for any Java version prior to the removal version or > > upwards from there. I don't think that this should be addressed by > tooling > > if keeping deprecated skeletons of the API can so easily avoid this > entire > > problem for all libraries without the need to chase down maintainers. > > > > Therefore, I really think that the SecurityManager and AccessController > > APIs should remain as skeletons and be deprecated, but not be marked > > forRemoval, especially without a clear roadmap for the actual removal > > forward. And while I appreciate the clean up effort - I do think the > > SecurityManager deprecation and feature removal is a right decision - I > > find the attempt to remove this API will cause unnecessary breakage and > > cause thousands of libraries to become unlinkable on future VM, without a > > clear need for it. Discovering this breakage would also require manually > > scanning the content of each library and affect all the big names in the > > industry. This would require big waves of dependency updates, where such > > updates sometimes will be impossible if only a single (transitive) > > dependency has not catched up, including major names such as Spring, > > Hibernate or Mockito. From experience, such major updating waves are > often > > complex and therefore avoided, which will hinder adoption of future JVM > > versions. This seems like a very high price to pay which could be easily >
Re: JEP 411: Deprecation with removal would break most existing Java libraries
One example for a currently necessary "doPrivileged" are Java agents where a class loading triggers agent code where the agent shares the stack with any code that loads a class for the first time. Otherwise, Byte Buddy wraps anything that might require privileges as privileged action to allow setting a policy that gives Byte Buddy for example access to class loaders, system properties or other things that the security manager currently checks. There's many uses of the security manager throughout the library, in the spirit of the API's invention. I could, of course, rip this code out of the library. But this would make it impossible for users that choose to use the functionality for now to update their dependency. This would certainly hinder a smooth transition as library maintainers will always have people drag at both ends of the JDK version range. After all, Java 8 is supported for another decade. Multi-release jars are neither a feasible option. They are not globally supported by all class loaders, and would require me to add a copy of an adjusted class file for any Java version prior to the removal version or upwards from there. I don't think that this should be addressed by tooling if keeping deprecated skeletons of the API can so easily avoid this entire problem for all libraries without the need to chase down maintainers. Therefore, I really think that the SecurityManager and AccessController APIs should remain as skeletons and be deprecated, but not be marked forRemoval, especially without a clear roadmap for the actual removal forward. And while I appreciate the clean up effort - I do think the SecurityManager deprecation and feature removal is a right decision - I find the attempt to remove this API will cause unnecessary breakage and cause thousands of libraries to become unlinkable on future VM, without a clear need for it. Discovering this breakage would also require manually scanning the content of each library and affect all the big names in the industry. This would require big waves of dependency updates, where such updates sometimes will be impossible if only a single (transitive) dependency has not catched up, including major names such as Spring, Hibernate or Mockito. From experience, such major updating waves are often complex and therefore avoided, which will hinder adoption of future JVM versions. This seems like a very high price to pay which could be easily avoided by only keeping a handful of skeleton classes. Am Mo., 14. Juni 2021 um 07:55 Uhr schrieb Alan Bateman < alan.bate...@oracle.com>: > cc'ing security-dev as that is the mailing list to use for this JEP. > > This JEP is the first of several in a multi-release/multi-year effort. > It's way too early to give any guess as to when the APIs will be > removed. As the JEP says, future releases may degrade the SM APIs so > that System.getSM returns always returns null or > AccessController::doPriv just runs the action. This should mean that > libraries that are compiling to older releases should continue to > compile and run on those releases. When they run on some future release > that degrades the implementation then it will be as if there is no SM. > So I would say the impact is little to none for libraries for the > foreseeable future. > > -Alan > > > On 13/06/2021 21:28, Rafael Winterhalter wrote: > > I am currently looking into how I should address JEP 411 in my library > Byte > > Buddy and I find it rather challenging. The problem I am facing is that I > > know of several users who rely on the security manager in their Java 8/11 > > applications. I would like to continue to support those users' use cases > as > > long as I support Java versions that contain the security manager, which > > will be for many years to come. At the same time, I would like to address > > the announced removal of the API and make sure that Byte Buddy can work > > without it prior to the deadline when the library in its current state > > would no longer link. > > > > From my understanding of the intention of JEP 411, the API was supposed > to > > be stubbed – similar to Android’s stubbing of the API - rather than being > > removed. However, with the announced deprecation for removal of > > AccessController and SecurityManager, I understand that I would need to > > fully remove the dispatching to work with future Java versions. > > > > Furthermore, it is difficult to create a working facade for dispatching > to > > the security manager only if it is available. Methods like > > AccessController.doPrivileged are caller sensitive and by adding a > utility > > to a library, this utility would leak to any potential user. It would > > therefore require package-private dispatchers for any relevant package, > > which would lead to a lot of copy-paste to retain backwards compatibility > > (given that a library cannot assume to be run as a module). > > > > Finally, removing the API would mean that Byte Buddy versions of the last > > ten years would no
Re: JEP 411: Deprecation with removal would break most existing Java libraries
On 14/06/2021 6:37 pm, Alan Bateman wrote: There are some libraries where the maintainers have put effort into working with a SM. Yes, I am one of them, very much so. At first it's a shock, but the show must go on, it could be an opportunity to address some long standing issues also. If Permission implementations are unmaintained, we are better off without them too and re implementing our own. It doesn't matter what they change to, I can generate my policy files. AccessController, AccessControlContext and DomainController are bigger fish, which will take longer, personally I wouldn't remove these classes, but it's not my choice. -- Regards, Peter.
Re: JEP 411: Deprecation with removal would break most existing Java libraries
The JEP addresses this: > In future JDK releases, we will degrade the Security Manager APIs so that > they remain in place but have limited or no functionality. ... This will allow libraries that support the Security Manager and were compiled against previous Java releases to continue to work without change or even recompilation. We expect to remove the APIs once the compatibility risk of doing so declines to an acceptable level. If your question is, when only few codebases will refer to the API and the compatibility impact is low enough for it to be removed, what if some of those few libraries still want to support versions prior to the removal, the answer is the same as with all removals (MR-JARs, or multiple artefacts). By definition, the impact of complete removal when the impact is low, would be low. — Ron > On 14 Jun 2021, at 09:38, Rafael Winterhalter wrote: > > One example for a currently necessary "doPrivileged" are Java agents where > a class loading triggers agent code where the agent shares the stack with > any code that loads a class for the first time. Otherwise, Byte Buddy wraps > anything that might require privileges as privileged action to allow > setting a policy that gives Byte Buddy for example access to class loaders, > system properties or other things that the security manager currently > checks. There's many uses of the security manager throughout the library, > in the spirit of the API's invention. > > I could, of course, rip this code out of the library. But this would make > it impossible for users that choose to use the functionality for now to > update their dependency. This would certainly hinder a smooth transition as > library maintainers will always have people drag at both ends of the JDK > version range. After all, Java 8 is supported for another decade. > Multi-release jars are neither a feasible option. They are not globally > supported by all class loaders, and would require me to add a copy of an > adjusted class file for any Java version prior to the removal version or > upwards from there. I don't think that this should be addressed by tooling > if keeping deprecated skeletons of the API can so easily avoid this entire > problem for all libraries without the need to chase down maintainers. > > Therefore, I really think that the SecurityManager and AccessController > APIs should remain as skeletons and be deprecated, but not be marked > forRemoval, especially without a clear roadmap for the actual removal > forward. And while I appreciate the clean up effort - I do think the > SecurityManager deprecation and feature removal is a right decision - I > find the attempt to remove this API will cause unnecessary breakage and > cause thousands of libraries to become unlinkable on future VM, without a > clear need for it. Discovering this breakage would also require manually > scanning the content of each library and affect all the big names in the > industry. This would require big waves of dependency updates, where such > updates sometimes will be impossible if only a single (transitive) > dependency has not catched up, including major names such as Spring, > Hibernate or Mockito. From experience, such major updating waves are often > complex and therefore avoided, which will hinder adoption of future JVM > versions. This seems like a very high price to pay which could be easily > avoided by only keeping a handful of skeleton classes. > > Am Mo., 14. Juni 2021 um 07:55 Uhr schrieb Alan Bateman < > alan.bate...@oracle.com>: > >> cc'ing security-dev as that is the mailing list to use for this JEP. >> >> This JEP is the first of several in a multi-release/multi-year effort. >> It's way too early to give any guess as to when the APIs will be >> removed. As the JEP says, future releases may degrade the SM APIs so >> that System.getSM returns always returns null or >> AccessController::doPriv just runs the action. This should mean that >> libraries that are compiling to older releases should continue to >> compile and run on those releases. When they run on some future release >> that degrades the implementation then it will be as if there is no SM. >> So I would say the impact is little to none for libraries for the >> foreseeable future. >> >> -Alan >> >> >> On 13/06/2021 21:28, Rafael Winterhalter wrote: >>> I am currently looking into how I should address JEP 411 in my library >> Byte >>> Buddy and I find it rather challenging. The problem I am facing is that I >>> know of several users who rely on the security manager in their Java 8/11 >>> applications. I would like to continue to support those users' use cases >> as >>> long as I support Java versions that contain the security manager, which >>> will be for many years to come. At the same time, I would like to address >>> the announced removal of the API and make sure that Byte Buddy can work >>> without it prior to the deadline when the library in its current state >>> would no longer link.
Re: JEP 411: Deprecation with removal would break most existing Java libraries
Binary compatibility only? Security.getSecurityManager() always returns null. Security.setSecurityManager() always throws a SecurityException (compatible because existing SecurityManager is allowed to prevent the call from succeeding). SecurityManager constructor always throws a SecurityException (compatible because existing SecurityManager is allowed to prevent the call from succeeding). Remove all methods from SecurityManager, no instances will be created, so methods will not be resolved, still binary compatible. For Policy, retain methods and constructor, throwing appropriate exceptions, eg SecurityException or NoSuchAlgorithmException. Note that constructor will throw no exception, the exception will be thrown when attempting to set Policy. Remove PolicyFile implementation. Leave PolicySPI as is, but don't load them. The first step would be to provide low level JDK hooks that allow for equivalent functionality to be implemented, back-porting to all long term releases, while leaving SecurityManager unchanged. Then change SecurityManager prior to next LTS version (Java 23) to binary compatible only. Then prior to the next LTS version (Java 29), remove it. Maintained software will have had plenty of time to update and be compatible across all supported Java releases, provided appropriate JDK hooks are provided, only unmaintained software will still require it and unmaintained software will still run on unmaintained Java releases in VM's with unmaintained OS's. Peter. On 14/06/2021 6:37 pm, Alan Bateman wrote: On 14/06/2021 08:35, Peter Firmstone wrote: I wouldn't want to see SecurityManager and Policy be neutralized, it's better to remove it and fail early so people update their software, there's a risk they may update without realizing it's no longer fully functional. Get rid of the baggage so people can start fresh with better practices. I think the context for the question is libraries that want to be able to compile to an older JDK release and work with a very wide range of JDK releases. Many libraries do not play well with a SM. They don't execute actions that require permission checks in privileged blocks and will often need to be granted AllPermission. This may have knock on impact to the components that call into these libraries, maybe they end up needing to be granted AllPermission too. Add callbacks or hand-off between threads to the picture and it can become farcical. If the library code isn't calling System.getSM or invoking AccessController.doPrivileged then it probably won't care if these APIs are degraded or removed. There are some libraries where the maintainers have put effort into working with a SM. Code in the library may use System.getSM, or doPriv or limited-doPriv. It may document the permissions that it requires and be helpful to someone assembling an application and creating its policy file. This JEP is mildly disruptive in that there will be warnings at compile-time or testing JDK 17+. If some future JDK releases degrades some of these APIs then it may be a bit more disruptive, maybe tests that try to set a SM will fail or need to be skipped. It might be that a library uses an exotic API that doesn't degrade in a sensible way and maybe that will be a bit more disruptive. Further out again, if the APIs are actually removed then it will be disruptive for libraries that want to support the possibility of being deployed with a SM on an older release. That may require some refactoring and the use of a MR-JAR as Remi mentioned. -Alan
Re: JEP 411: Deprecation with removal would break most existing Java libraries
On 14/06/2021 08:35, Peter Firmstone wrote: I wouldn't want to see SecurityManager and Policy be neutralized, it's better to remove it and fail early so people update their software, there's a risk they may update without realizing it's no longer fully functional. Get rid of the baggage so people can start fresh with better practices. I think the context for the question is libraries that want to be able to compile to an older JDK release and work with a very wide range of JDK releases. Many libraries do not play well with a SM. They don't execute actions that require permission checks in privileged blocks and will often need to be granted AllPermission. This may have knock on impact to the components that call into these libraries, maybe they end up needing to be granted AllPermission too. Add callbacks or hand-off between threads to the picture and it can become farcical. If the library code isn't calling System.getSM or invoking AccessController.doPrivileged then it probably won't care if these APIs are degraded or removed. There are some libraries where the maintainers have put effort into working with a SM. Code in the library may use System.getSM, or doPriv or limited-doPriv. It may document the permissions that it requires and be helpful to someone assembling an application and creating its policy file. This JEP is mildly disruptive in that there will be warnings at compile-time or testing JDK 17+. If some future JDK releases degrades some of these APIs then it may be a bit more disruptive, maybe tests that try to set a SM will fail or need to be skipped. It might be that a library uses an exotic API that doesn't degrade in a sensible way and maybe that will be a bit more disruptive. Further out again, if the APIs are actually removed then it will be disruptive for libraries that want to support the possibility of being deployed with a SM on an older release. That may require some refactoring and the use of a MR-JAR as Remi mentioned. -Alan
Re: JEP 411: Deprecation with removal would break most existing Java libraries
My thoughts on how to proceed with this is: 1. Develop authorization layer security provider services in OpenJDK, back port it to Java 8 and Java 11 (these provide most of the utilised functionality of SecurityManager, allowing developers to only implement those which they need, without enabling SecurityManager and editing policy files). 2. Remove SecurityManager, Policy and Policy provider in OpenJDK 19. 3. Leave AccessController and AccessControlContext in place, provide a security property that allows the stack walk to be disabled (One ProtectionDomain to represent all code in JVM, without any permissions, later making it the default, while allowing it to be enabled) and continue to inject user Subject principles using SubjectDomainCombiner for JAAS compatibility. Developers can implement just the authorization they feel necessary for users and develop their own configuration, or adapt existing. 4. At some point, preferably when StackWalker has equivalent performance, replace the internals of AccessController. Gut feel is that removal of AccessController, AccessControlContext and DomainCombiner causes carnage with JAAS, it's a lot to deal with, SecurityManager and Policy, I don't need them if I've got authorization layer hooks in the JVM. I wouldn't want to see SecurityManager and Policy be neutralized, it's better to remove it and fail early so people update their software, there's a risk they may update without realizing it's no longer fully functional. Get rid of the baggage so people can start fresh with better practices. Note that I'll be implementing a full authorization layer based on least privilege principles, that still utilizes a stack walk. I won't use the standard Java Permission implementations, as I'd like to do things a little differently, for usability reasons, this will do everything I need at least. If the authorization layer is back ported, then I can support all current versions. My policy files are generated, so I don't need permission implementations to remain compatible. Whatever happens with JAAS will need to be backported, so we can support all versions. Regards, Peter. On 14/06/2021 3:54 pm, Alan Bateman wrote: cc'ing security-dev as that is the mailing list to use for this JEP. This JEP is the first of several in a multi-release/multi-year effort. It's way too early to give any guess as to when the APIs will be removed. As the JEP says, future releases may degrade the SM APIs so that System.getSM returns always returns null or AccessController::doPriv just runs the action. This should mean that libraries that are compiling to older releases should continue to compile and run on those releases. When they run on some future release that degrades the implementation then it will be as if there is no SM. So I would say the impact is little to none for libraries for the foreseeable future. -Alan On 13/06/2021 21:28, Rafael Winterhalter wrote: I am currently looking into how I should address JEP 411 in my library Byte Buddy and I find it rather challenging. The problem I am facing is that I know of several users who rely on the security manager in their Java 8/11 applications. I would like to continue to support those users' use cases as long as I support Java versions that contain the security manager, which will be for many years to come. At the same time, I would like to address the announced removal of the API and make sure that Byte Buddy can work without it prior to the deadline when the library in its current state would no longer link. From my understanding of the intention of JEP 411, the API was supposed to be stubbed – similar to Android’s stubbing of the API - rather than being removed. However, with the announced deprecation for removal of AccessController and SecurityManager, I understand that I would need to fully remove the dispatching to work with future Java versions. Furthermore, it is difficult to create a working facade for dispatching to the security manager only if it is available. Methods like AccessController.doPrivileged are caller sensitive and by adding a utility to a library, this utility would leak to any potential user. It would therefore require package-private dispatchers for any relevant package, which would lead to a lot of copy-paste to retain backwards compatibility (given that a library cannot assume to be run as a module). Finally, removing the API would mean that Byte Buddy versions of the last ten years would no longer link in future JDKs. For Byte Buddy where new Java versions often require an update, that might not be a big issue but many other libraries do support the API, I don’t feel it would be a rather severe restriction and cause unnecessary breakage if API is removed, rather than stubbed. I am thinking of libraries like Netty here which are rather omnipresent and would suddenly no longer link, a concept that is unlikely
Re: JEP 411: Deprecation with removal would break most existing Java libraries
cc'ing security-dev as that is the mailing list to use for this JEP. This JEP is the first of several in a multi-release/multi-year effort. It's way too early to give any guess as to when the APIs will be removed. As the JEP says, future releases may degrade the SM APIs so that System.getSM returns always returns null or AccessController::doPriv just runs the action. This should mean that libraries that are compiling to older releases should continue to compile and run on those releases. When they run on some future release that degrades the implementation then it will be as if there is no SM. So I would say the impact is little to none for libraries for the foreseeable future. -Alan On 13/06/2021 21:28, Rafael Winterhalter wrote: I am currently looking into how I should address JEP 411 in my library Byte Buddy and I find it rather challenging. The problem I am facing is that I know of several users who rely on the security manager in their Java 8/11 applications. I would like to continue to support those users' use cases as long as I support Java versions that contain the security manager, which will be for many years to come. At the same time, I would like to address the announced removal of the API and make sure that Byte Buddy can work without it prior to the deadline when the library in its current state would no longer link. From my understanding of the intention of JEP 411, the API was supposed to be stubbed – similar to Android’s stubbing of the API - rather than being removed. However, with the announced deprecation for removal of AccessController and SecurityManager, I understand that I would need to fully remove the dispatching to work with future Java versions. Furthermore, it is difficult to create a working facade for dispatching to the security manager only if it is available. Methods like AccessController.doPrivileged are caller sensitive and by adding a utility to a library, this utility would leak to any potential user. It would therefore require package-private dispatchers for any relevant package, which would lead to a lot of copy-paste to retain backwards compatibility (given that a library cannot assume to be run as a module). Finally, removing the API would mean that Byte Buddy versions of the last ten years would no longer link in future JDKs. For Byte Buddy where new Java versions often require an update, that might not be a big issue but many other libraries do support the API, I don’t feel it would be a rather severe restriction and cause unnecessary breakage if API is removed, rather than stubbed. I am thinking of libraries like Netty here which are rather omnipresent and would suddenly no longer link, a concept that is unlikely intuitive to a lot of developers. Therefore, my question is: should SecurityManager, AccessController and the Policy APIs really be deprecated for removal? Rather, I think that the APIs should be deprecated, but be retained with stubbed implementations. System.getSecurityMananger would then always return null. System.setSecurityManager on the other hand could be deprecated for removal. This way, existing code could continue to work as if the security manager is not active, which already is the common scenario and would not cause any disruption at the small price of keeping a handful of some stubbed classes. Thanks for advice on how this is intended to be handled by library developers like me. Best regards, Rafael
