Re: Problem with multi-release jars, agents and the bootstrap class path
On 13/10/2016 21:43, Andrew Dinn wrote: : I am aware that the code I add to the bootstrap can only rely on a core subset of the JDK runtime classes being available. That ought not to be an issue -- I have deliberately kept the agent's footprint as small as possible because every time Byteman uses an API it makes it trickier to instrument the API class. I'd be interested to know what you are hoping to exclude though, just in case. Am I ok with print stream? file streams? localhost server sockets? Just checking :-) The granularity is module. It would be great if we could somehow get to the point where the only module defined to the boot loader is java.base but that is a long way off (java.desktop is the biggie that would need to move before its dependences can move). If it were just java.base then the "core" APIs would be visible, this includes file and networking. : Yes, by hoisting I mean precisely that. The minor wrinkle that the class which adds the agent jar to the bootstrap path actually gets loaded from that same jar (but via the system classpath) is what led me to use the term 'hoist'. There is probably a joke in here somewhere about 'my own petard' but I'll demur. LOL but yes, major hazards here as presumably you have types of the same name defined to both the system and boot loaders. : It's a shame multi-release won't work for all agents but never mind. I see Paul has updated the text in JEP 238 to make this clear. -Alan
Re: Problem with multi-release jars, agents and the bootstrap class path
On 13/10/16 18:27, Alan Bateman wrote:
> What you see is deliberate. It wouldn't be impossible for the boot
> loader to support MR JARs but it hardly seems worth the complexity. One
> reason is that it's not a common way to deploy a general purpose
> library. Even if it were then it would be problematic going forward as
> we reduce the number of modules defined to the boot loader (you can't
> instrument code in core modules to call into a SQL library for example
> because java.sql types are not visible to code in the boot loader). The
> other thing is that the boot class path as we used to know it is mostly
> gone, it only really exists when using -Xbootclasspath/a or when JVM TI
> or java.lang.instrument do the equivalent.
Ok, that's what I was expecting to be the answer :-)
I am aware that the code I add to the bootstrap can only rely on a core
subset of the JDK runtime classes being available. That ought not to be
an issue -- I have deliberately kept the agent's footprint as small as
possible because every time Byteman uses an API it makes it trickier to
instrument the API class. I'd be interested to know what you are hoping
to exclude though, just in case. Am I ok with print stream? file
streams? localhost server sockets? Just checking :-)
> For the "hoist"-ing scenario then I assume you mean
> Instrumentation::appendToBootstrapClassLoaderSearch to add supporting
> classes to the boot class path so that they are visible to instrumented
> code. Can you use the same method to append a JAR file containing
> 53.0/9+ classes before you append the other JAR. That should be
> equivalent to what a MR JAR would give you, albeit without the
> convenience of everything in one artifact.
Yes, by hoisting I mean precisely that. The minor wrinkle that the class
which adds the agent jar to the bootstrap path actually gets loaded from
that same jar (but via the system classpath) is what led me to use the
term 'hoist'. There is probably a joke in here somewhere about 'my own
petard' but I'll demur.
I could indeed use two jar files -- in fact my last attempt at a JDK9
compatible implementation did precisely that. However, that actually
turns out to be /much/ more inconvenient because it means the agent jar
doing the hoisting needs to be able to work out where the auxiliary jar
is. This results in a lot of configuration grief (because of the need to
cope with command line vs dynamic install and also to make it easy to do
from tools like maven). So I much prefer a one jar solution.
As it happens, I have managed to encapsulate the JDK8- vs JDK9+ variant
functionality behind an interface. This means that I ham able to employ
just one setup class which needs a JDK8- and JDK9+ variant -- it's job
is to provide a static method which populates the interface with a
suitable implementation. So, I /can/ actually resolve this issue by
bundling all the code in one jar but with this variant implemented as
two distinct classes. I will simply load the relevant variant by name +
reflectively invoke it's static setup method conditional on whether or
not the JDK includes Jigsaw.
I was asking not because this problem is not resolvable but because
I thought the multi-release option was a very neat solution that I
really ought to try to use
I envisaged that other agent implementors might not find it so easy to
encapsulate the required functionality
It's a shame multi-release won't work for all agents but never mind.
Thanks very much for the quick and helpful response.
regards,
Andrew Dinn
---
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander
Re: Problem with multi-release jars, agents and the bootstrap class path
On 13/10/2016 17:40, Andrew Dinn wrote: : A Few Questions: Am I right that this is a deliberate choice of bootstrap loader behaviour? Or is there perhaps some other bug here? If this is a deliberate choice is there any possibility of reviewing it? Would it be particularly difficult to modify the boot loader to recognise jars in multi-release format? What you see is deliberate. It wouldn't be impossible for the boot loader to support MR JARs but it hardly seems worth the complexity. One reason is that it's not a common way to deploy a general purpose library. Even if it were then it would be problematic going forward as we reduce the number of modules defined to the boot loader (you can't instrument code in core modules to call into a SQL library for example because java.sql types are not visible to code in the boot loader). The other thing is that the boot class path as we used to know it is mostly gone, it only really exists when using -Xbootclasspath/a or when JVM TI or java.lang.instrument do the equivalent. For the "hoist"-ing scenario then I assume you mean Instrumentation::appendToBootstrapClassLoaderSearch to add supporting classes to the boot class path so that they are visible to instrumented code. Can you use the same method to append a JAR file containing 53.0/9+ classes before you append the other JAR. That should be equivalent to what a MR JAR would give you, albeit without the convenience of everything in one artifact. -Alan
Problem with multi-release jars, agents and the bootstrap class path
Disclaimer:
Although this post relates to handling of multi-release jars I'm
cross-posting to jigsaw-dev as well as core-libs-dev. That's because the
problem arises only because my JVMTI Java agent now needs to operate in
dual mode to cope with Jigsaw when deployed on JDK9+ and to avoid
reference to Jigsaw when deployed on JDK8-, hence the desire to use the
multi-release format. Apologies if the cross-post is redundant.
The Problem:
I have re-packaged my Byteman JVMTI Java agent using a multi-release jar
format so that it can operate as is on JDK8- (compiled as target 6 and
no reference to Jigsaw code) while providing revised functionality on
JDK9 that accounts for the presence of modules. This works fine if I
deploy the agent on the system classpath.
However, for some modes of operation my agent Main class hoists the
agent jar into the bootstrap classpath. This is needed so that I can
inject references to agent classes into JDK bootstrap code like Thread
etc. The problem is that bootstrap loader does not appear to recognise
that the jar is in multi-release format.
I can understand that this may seem to be an appropriate behaviour under
the assumption that code in the bootstrap ought to be the JDK runtime
code and therefore ought to be compiled at the current JDK/JVM version.
However, that assumption doesn't really hold when the code in question
is an agent which may well need to operate on old and new JDK versions.
A Few Questions:
Am I right that this is a deliberate choice of bootstrap loader
behaviour? Or is there perhaps some other bug here?
If this is a deliberate choice is there any possibility of reviewing it?
Would it be particularly difficult to modify the boot loader to
recognise jars in multi-release format?
Thanks in advance!
regards,
Andrew Dinn
---
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander
