On 14.07.2016 23:24, Alan Bateman wrote:
On 14/07/2016 21:20, Jochen Theodorou wrote:
What I would wish for at this moment is a document explaining the
runtime aspects of the module system, including limitations reflection
and normal method calls, as well as things like layers / runtime
generated modules - as well as the methods to be used to create and
change these. I don´t anything JLS ready here and just the behaviour
of the one of the latest releases is also good. I cannot give feedback
on a design I do not know. Because maybe some frameworks are content
with the compile time side of this, I am not.
Jochen - have you worked through the documents, slides and recordings
that are linked from the main Project Jigsaw page [1]?
I know in other threads we we alluded to slides on how
Nashorn/Javascript has been updated to work with modules (and spin
modules at runtime). I realize Groovy is different but I think we should
do this as it may be useful to other language implementers too.
[1] http://openjdk.java.net/projects/jigsaw/
Well, I was hoping for a document, videos do not work well for me and
slides are better, but often useless without the actual talk... depends
of course
The problem is that this is a lot of material that barely scratches the
points I am wondering about.
Example http://openjdk.java.net/projects/jigsaw/talks/
Javaone 2015
Prepare for JDK 9 -> for me useless
Introduction to Modular Development -> for runtime aspects useless
Advanced Modular Development -> mentions addReads on slide 36 and is
done with the runtime aspect with that basically.
Project Jigsaw: Under the Hood -> is more interesting from slide 34 on,
but does not answer my questions enough.
Project Jigsaw Hack Session -> the whole thing as video
Devoxx BE 2015 is only video
I will give you an analysis of my situation so far:
assuming the whole Groovy runtime is a module, and assuming I have other
modules as well, and then I want to use a Groovy script at runtime, to
access those modules. And let us assume those modules do not know of
Groovy. Let us further assume the Groovy runtime will be in a loader
that delegates to a loader knowing the classes of those modules, or is
in the same loader.
So first problem is ... can I use the unnamed module for the scripts?
They read every (named) module, thus they can access the types.
Next are 4 types of method invocations to consider for the case of an
invocation from script in the unnamed module to named module:
reflective, indy, direct, generated bytecode for callsites (I name this
one csgen in short). The method itself is looked up by reflection
* for indy it is only a question of the lookup object, which will be
provided by the script and can do the call
* direct calls will work
* reflection would be done from the groovy runtime, thus the runtime
need to have a read on the named module. That´s the first time I will
need Java9 special code in this scenario
* csgen is more unclear. Let us assume it is in the unnamed package as
well, in a loader that delegates to the script loader. Basically it
becomes a direct call and works.
Next step is for my Groovy runtime having to call a method from the
script. Since everything is exported there is no problem on this side.
* indy will work after adding the read
* reflection - same
* direct call - same
* cs gen... the call to the csgen method will be done by an
interface/class of the Groovy runtime, and since the csgen class is in
the unnamed package, it has no problem calling the script methods.
Let us look at the same scenario with a named module for scripts. I will
use that as a step in between to think about actual compile time modules
written in Groovy. So for simplicity I assume I can create such a module
at runtime and it will export everything. How to do that? I have not the
slightest idea. Anyway, that means now I have calls from named script
module to named module:
* indy will work after the script module added a read to the target
module. This will require in the script, with a direct call to addReads,
because of caller sensitivity. Which means instead of just having JDK9
specific code in my runtime, I will now need JDK9 specific code to my
bytecode as well. And this needs to happen before any normal method
invocation can take place, thus static initializer, first thing... which
means completely new code for the compiler, that will work only on JDK9.
This means the compiler will then have to at least know if he compiles
for JDK9 or not...
* direct call - same
* reflection - the call is effectively done from the Groovy runtime, so
the Groovy runtime will have to add the read.
* csgen in the unnamed module means.. since the call is gated by an
interface/class from the runtime I assume I do not need a read edge from
the script module to the unnamed module csgen is in. The csgen class
then will be able to read the exported class of the target module, thus
has no problem.
Calls from the groovy runtime to the script are not different to before.
Next step is then precompiled Groovy code in a named module. The
difference to before is that we now have to look at calls from hidden
API to hidden API of the same module and we have compile time defined
requires and exports.
Since the Groovy runtime needs to be able to call some methods by
reflection all packages, including the hidden API, will have to be
exported to the groovy runtime and of course there will be a require for
the Groovy runtime.
Let us briefly look at a call from the script module to an exported
class in another named module again... the script module will now
require the other named module, which means we have a reads here already.
* direct call works
* reflection - runtime has to add read
* indy works
* csgen in the unnamed module... the call is again done using
interfaces/classes from the runtime, meaning it should work.
Then let us look at a call from hidden/public API to hidden API of the
same module of our precompiled script.
* direct call works
* indy works
* reflection works, since everything is exported to the groovy runtime..
* csgen in the unnamed module... while the call to csgen will work
(gated by classes/interfaces of the runtime), the call from csgen to
hidden API will fail. Adding a read won´t help, end of line. A named
module would have the same problem. Solution unclear.
As for calls from the groovy runtime to the hidden API of the script
module... Some csgen problem, everything else is fine.
Finally calls from the script module to the hidden API of another
module... I can make this sometimes work with indy and with reflection.
csgen will have the known problem, direct calls will not work. Of course
there is now the question of if I need such a call... assume you did
write a program like this:
Module A, export exported
package exported
class MyExportedList extends ArrayList {
def sum(init){
def ret = init
for (it in this) ret +=it
return ret
}
}
Module B, hidden API, require A
class HiddenX {
int value
def plus(HiddenX other) { return value+other.value }
}
def list = [new HiddenX(value:1), new HiddenX(value:2), new
HiddenX(value:3)] as MyExportedList
assert list.sum(new HiddenX(value:0)) == 6
This will require the sum method in Module A to call plus on HiddenX for
the "+=". There is no interface or class for this, just the public
method. Since the class is in the hidden API of B, A will not be able to
access it.
* A direct method call is out of question for this of course, so no
static case at all.
* reflection through the groovy runtime
* indy... before I would have used the lookup object of the caller to
realize the call. Since A cannot read HiddenX, this won´t work anymore.
I would have to resort to a private "everything is allowed" constructor
in Lookup. Basically a hack, of which I won´t know if it will work in
the next JDK version. Maybe I could "steal" a lookup object from HiddenX
to avoid this, but it is very allowed to write HiddenX in another
language like Java, and then I will not be able to produce a helper
method for this. Meaning I will have to stay with the hackish
maintenance nightmare of using a private constructor.
* csgen... the sad story continues.
And btw, how am I supposed to decide if this call is allowed or not?
To sum things up:
* module runtime creation and handling beyond adding reads is unclear
* setAccessible failing on public classes from hidden APIs is
inconvenient, but at least we can find code for this, which is
compatible for multiple JDK versions
* csgen will probably have to fall back to reflection for most cases,
which means it will be a lot slower. A bigger change would be to give it
an indy backend. We would probably still suffer a performance penalty,
since we then have an additional layer with wrapping in between, that
cannot be removed by the JIT, as long as the JIT is unable to look
beyond the indy callsite itself.
* the indy part will have to use "forbidden" API now.
* runtime scripts will go into the unnamed module
* hidden APIs and non-public modifiers will be ignored by Groovy for as
long as the "forbidden" API works, which keeps current semantics
* most of the groovy runtime makes still heavy use of reflection and
will probably have to be reworked to use indy everywhere, if on JDK9
(probably JDK8+).
And I am not happy with this outcome so far.
bye Jochen
