Summary Regarding the proposed solution of adding method redefineModule to class Instrumentation:
I have a Proof Of Concept implementation which shows that that this solution works for my agent and does not impose unworkable restrictions on its operation or use. The implementation can be generalised so that it will very likely work just as well for other agents -- indeed, the current implementation has already been implemented for the most part to address this general use case. The current implementation involves some clumsiness interfacing the agent to the additional code needed for the solution when (as will likely be the case for most existing agents) the agent code needs to be compiled to target pre-Jigsaw JDK runtimes. I am still working on the current implementation to make it easier to use, to fully document how it can be used and how it works and to package it so it can employed by other agents. More details of of what has been achieved (and how) plus the location of the implementation and some of the changes I need to make are included below in case they are of wider interest. regards, Andrew Dinn ----------- What Does This POC Prove? I have successfully modified my agent to achieve reflective access to unexported packages using the redefineModule method of class Instrumentation. This modification satisfies all the desired design goals: 1) Decoupling from non-Jigsaw operation: the agent jar retains all the functionality required to operate on non-Jigsaw JDKs (it is compiled target 6) all functionality required to operate on a Jigsaw-enabled JDK is located in a single supplementary jar (it is compiled target 9 and is /not/ a module jar) 2) Simple deployment: when deploying the agent on the command line full reflective access under Jigsaw can be achieved either by adding the supplementary jar to the system classpath or by requesting the agent to add it to the classpath at install time (using method appendToSystemClassLoaderSearch of class Instrumentation) when deploying the agent dynamically the supplementary jar can be installed as needed by requesting the agent to add it to the classpath at install time (using method appendToSystemClassLoaderSearch of class Instrumentation) the convenience scripts used to manage both JVM-startup deployment (bmjava) and dynamic deployment (bminstall) of the agent have been supplemented with a simple command option to control installation of the supplementary jar the annotations used to configure auto-loading of the agent as part of JUnit/TestNG testing can be (but have not yet been) extended with a simple configuration field to control installation of the supplementary jar 3) Fallback operation: if the supplementary jar is not deployed the agent defaults to being able to access exported non-public classes + members -- it will throw an exception when an attempt is made to access unexported non-public classes + members 4) No prejudice to existing module visibility the supplementary jar uses the Layer API to create a custom module used as target for all exports installed by calls to redefineModules, ensuring that module accessibility for all other (JDK or application) modules is unaffected by the presence of the agent n.b. the custom module is /not/ populated from a disk-based module jar -- instead the agent must provide the module's classloader with byte[] class definitions on demand (the expectation is that the agent will synthesise the required byte[] definitions using ASM but it can arrange to load definitions from disk if it wishes), the class load process being driven by the agent by calling the classloader's loadClass method. 5) Controlled access to enabled Members the sole class installed into the custom module by /my/ agent provides the agent (and only the agent) with a method which will call setEnabled for a specific Member on demand, if necessary first establishing whatever export relation is required from the module of the Member's owner class Where Is The Implementation? The current implementation is a full implementation of the functionality required by my agent. It is almost in a state where other agents could pick up and use the supplementary jar to provide themselves with a similar capability. The code still needs a small amount of cleaning up and quite a bit of javadoc needs to be added. There are also a few details that I am unsure about the correctness of (although they don't seem to inhibit it from working), most specifically the use of URLs in the ModuleFinder and ModuleReader. The code is located in my personal Byteman github repo in its own branch https://github.com/adinn/byteman/tree/jigsaw-enablement This branch builds the normal agent if you built with JDK{6-8}. It builds the Jigsaw capable agent if you build with Jigsaw release 9-ea+126. I have no idea what happens if you use the current trunk jdk9 to build it (it may be missing critical Jigsaw features so, basically, don't :-). The branch contains two extra maven modules not present in the master branch: jigsawlayer provides a means to hoist a custom module into the runtime without needing to add a module jar to the module path (in fact without needing a jar at all) jigsaw contains source for a class injected into the module by the agent -- the source is just expository since the agent actually synthesises an equivalent class, avoiding various problems which were encountered in trying to make the module classes available from a module jar on disk maven module agent includes some changes to the agent's Transformer class to configure Jigsaw operation plus a few minor changes to the agent code to ensure that Member accesses obtains a correctly enabled handle. What Code Was Added Or Changed? module jigsawlayer: Maven submodule jigsawlayer contains all the code which goes into the supplementary jar byteman-jigsawlayer.jar. Essentially, it provides the code used to create the custom module and leaves it up to the agent to synthesise classes which go into the module. The API is a single method JigsawLayer.installModule(String, String, Map<String, byte[]>) The String arguments are the name of the custom module and the name of a single package for the module to export. Ideally, the API should accept a list of exported package names and a list of modules to import (it currently hard wires import of java.instrument which my setEnabled caller needs) The Map provides a way for the agent to configure the module classloader so it can retrieve a byte[] definition for any given (String) classname. This avoids the need for the igsawlayer code to be involved in loading or synthesising bytecode. The ModuleReader used by the classloader calls the Map's get method when asked to load a class. In my case I provide a pre-populated HashMap but you can provide a Map implementation whose get method generates the required class definitons on demand. Ideally, the Map argument would have type Function<String, byte[]> but that doesn't work when the agent is compiled target JDK6. So, I settled for Map. Note also that it is not possible for the agent to define an equivalent to class Function for use in the module API to use because the custom module cannot see classes defined by the agent. [I am considering making the argument an Object and having the module API check for a Function -- in which case it casts it and uses it -- or a class which implements apply -- in which case it wraps it with a standard wrapper which implements Function and calls the apply method reflectively.] module agent: The agent checks during agent startup whether the runtime is Jigsaw-enabled (it uses a call to ClassLoader.loadClass() to detect whether platform class Module exists). If so it calls method Transformer.initForJigsaw(). This is where the jigsawlayer installModule API method gets called. Note that the installModule call is done reflectively because you cannot build the code as separate maven modules if you use direct linkage -- the agent code has to be compiled target JDK6 and the jigsawlayer API method has to be compiled target JDK9. The agent only needs to define one class, JigsawAccessEnabler, in the custom module so it passes a HashMap into the installModule call populated with a byte[] corresponding to the class definition. The code synthesised for this class (by method getJigsawAccessEnablerBytes() of agent class Transformer) is based on the source provided in (dummy) module jigsaw. The generating method (getJigsawAccessEnablerBytes) is annotated with the originating source lines. Having created the Map and installed the module the agent calls classloader.loadClass() to drive installation of class JigsawAccessEnabler into the custom module. It uses reflection to create an instance of this class. This instance is passed to each Byteman Rule where it can be used to enable access to Members. The JigsawAccessEnabler instance is given access to the agent's Instrumentation instance at create time, allowing it to export packages to it's own module as needed (n.b JigsawAccessEnabler throws an exception if it finds it is not located in a named module). jigsaw module: If Jigsaw is not available or the supplementary jar has not been configured the agent uses an instance of the default class DefaultAccessEnabler to enable access. Ideally DefaultAccessEnabler and JigsawAccessEnabler should both implement the common interface AccessEnabler used by class Rule. However, this is not possible because of the need for target JDK6/JDK9 deployment. An interface deployed in the agent jar cannot be seen (and hence used) from a class defined in the custom module. So, that explains why JigsawAccessEnabler does not implement AccessEnabler. The alternative of placing the interface in the supplementary jar or in the custom module is not an option either. They are both compiled target JDK9 so are of no use when the agent is deployed on JDK{6-8}. So, the agent has to wrap the JigsawAccessEnabler instance with an instance of a proxy class (JigsawAccessEnablerWrapper) which does implement AccessEnabler. The proxy operates by forwarding method invocations to the wrapped JigsawAccessEnabler using reflection.
