Re: Ping - Re: RFR 8078812, Test RMI with client and servers as modules
Stuart,
While I would agree on the use of one type, not two, for file paths, I
would question the choice to use String instead of Path. If something
is a file path, use the type system to say so, and use a Path.
-- Jon
On 05/20/2016 04:52 PM, Stuart Marks wrote:
On 5/16/16 1:18 AM, Felix Yang wrote:
please review the updated webrev, which remove not-suggested
filesystem
modification and codebase stuff:
http://cr.openjdk.java.net/~xiaofeya/8078812/webrev.02/
Hi Felix,
OK, this is looking much better and simpler. The big improvement is,
as we had discussed, the creation of a common fixture for the tests.
The tests then use the same fixture in different ways to exercise the
different cases. This makes this more easily extensible to additional
cases.
First to be done are a few easy cleanups:
* Since we're no longer using a registry, the starting and stopping of
the registry process in the start- and shutdownRMIRegistry() methods
is no longer necessary. They can be removed entirely, along with the
rmiRegistry field.
* The imports they required, along with some other unused import
statements, can also be removed.
* The fixture setup method, compileAll(), should be annotated
@BeforeTest instead of @BeforeMethod. Compiling once at the beginning
of the test should be sufficient.
With these out of the way, my observation is that it's still really
quite difficult to follow what the test is doing, particularly in
setting up the text fixture.
One reason for this is the repeated conversions between Path and
String. Some places need Paths and some need Strings. Path elements
are concatenated in some places with path.resolve(String) and string
concatenation with File.separator or File.pathSeparator in others. In
some cases, fields such as
MODS_DIR = Paths.get("mods")
are defined when in my opinion it'd just be better to use the string
literal "mods" in a couple places.
In any case I've taken the liberty of posting some cleanups to a
branch in the sandbox. I've written a couple utilities to concatenate
strings using File.separator and File.pathSeparator, and I've kept
things mostly as strings, only converting to Paths when absolutely
necessary. I've also renamed the directory of compiled classes as the
"exploded" directory since that's sort-of the descriptive term in use
for a hierarchy of individual class files.
The sandbox branch is JDK-8078812-branch and the diff from your webrev
can be viewed here:
http://hg.openjdk.java.net/jdk9/sandbox/jdk/rev/befcc172e68e
and the ModuleTest.java file (the only one I modified) can be viewed
here:
http://hg.openjdk.java.net/jdk9/sandbox/jdk/file/befcc172e68e/test/java/rmi/module/ModuleTest.java
It's up to you whether you want to accept my changes and continue from
this point, or go in a different direction, but to my eye this is
cleaner and easier to follow.
* * *
Now, finally, on to more substantive review issues. :-)
One thing that seemed to be missing was that the application itself
wasn't wrapped up into a jar file. I've added another Jar command that
does this.
Now we have the client, server, and app as separate hierarchies under
the "exploded" directory, and as modules under the "mods" directory.
I think the idea of having the "exploded" class hierarchy as well as
jar files useful as modules is a good one. This will let you add
cases, where different components are on the classpath or are loaded
as modules, in addition to the two already present here.
One issue here is that there's a module-info class for the app. This
makes the app an actual named module (I think) as opposed to an
automatic module like the client and server jar files. It seems like
it would be preferable to be consistent and have them all be automatic
modules.
Given this arrangement, it should be pretty easy to have tests for any
of the combinations we want of classpath vs modules. I guess there are
8 combinations: three components, each of which can be on the
classpath or as a module. It's not clear to me that we need all 8
combinations. It's probably sufficient to have a reasonable subset.
An idea for possible future expansion is to mix automatic modules with
named modules. I'm not entirely sure how to do that. Perhaps one way
is to have module-info files for all the components, and then create
variant jar files with the module-info.class omitted. That way we'd
have a modular jar, and then a "plain" jar (without module-info.class)
that'd be suitable for use as an automatic module or to be put on the
classpath. That'd be 3^3=27 combinations, which I certainly think is
overkill.
In any case, for this initial changeset, I think it's sufficient to
test a few combinations of automatic modules vs. classpath. We can
extend the cases to include named modules later. Please make a
recommendation for some set of combinations and implement it, and then
send it out for a final round of review.
Thanks
Re: Ping - Re: RFR 8078812, Test RMI with client and servers as modules
On 5/16/16 1:18 AM, Felix Yang wrote:
please review the updated webrev, which remove not-suggested filesystem
modification and codebase stuff:
http://cr.openjdk.java.net/~xiaofeya/8078812/webrev.02/
Hi Felix,
OK, this is looking much better and simpler. The big improvement is, as we had
discussed, the creation of a common fixture for the tests. The tests then use
the same fixture in different ways to exercise the different cases. This makes
this more easily extensible to additional cases.
First to be done are a few easy cleanups:
* Since we're no longer using a registry, the starting and stopping of the
registry process in the start- and shutdownRMIRegistry() methods is no longer
necessary. They can be removed entirely, along with the rmiRegistry field.
* The imports they required, along with some other unused import statements, can
also be removed.
* The fixture setup method, compileAll(), should be annotated @BeforeTest
instead of @BeforeMethod. Compiling once at the beginning of the test should be
sufficient.
With these out of the way, my observation is that it's still really quite
difficult to follow what the test is doing, particularly in setting up the text
fixture.
One reason for this is the repeated conversions between Path and String. Some
places need Paths and some need Strings. Path elements are concatenated in some
places with path.resolve(String) and string concatenation with File.separator or
File.pathSeparator in others. In some cases, fields such as
MODS_DIR = Paths.get("mods")
are defined when in my opinion it'd just be better to use the string literal
"mods" in a couple places.
In any case I've taken the liberty of posting some cleanups to a branch in the
sandbox. I've written a couple utilities to concatenate strings using
File.separator and File.pathSeparator, and I've kept things mostly as strings,
only converting to Paths when absolutely necessary. I've also renamed the
directory of compiled classes as the "exploded" directory since that's sort-of
the descriptive term in use for a hierarchy of individual class files.
The sandbox branch is JDK-8078812-branch and the diff from your webrev can be
viewed here:
http://hg.openjdk.java.net/jdk9/sandbox/jdk/rev/befcc172e68e
and the ModuleTest.java file (the only one I modified) can be viewed here:
http://hg.openjdk.java.net/jdk9/sandbox/jdk/file/befcc172e68e/test/java/rmi/module/ModuleTest.java
It's up to you whether you want to accept my changes and continue from this
point, or go in a different direction, but to my eye this is cleaner and easier
to follow.
* * *
Now, finally, on to more substantive review issues. :-)
One thing that seemed to be missing was that the application itself wasn't
wrapped up into a jar file. I've added another Jar command that does this.
Now we have the client, server, and app as separate hierarchies under the
"exploded" directory, and as modules under the "mods" directory.
I think the idea of having the "exploded" class hierarchy as well as jar files
useful as modules is a good one. This will let you add cases, where different
components are on the classpath or are loaded as modules, in addition to the two
already present here.
One issue here is that there's a module-info class for the app. This makes the
app an actual named module (I think) as opposed to an automatic module like the
client and server jar files. It seems like it would be preferable to be
consistent and have them all be automatic modules.
Given this arrangement, it should be pretty easy to have tests for any of the
combinations we want of classpath vs modules. I guess there are 8 combinations:
three components, each of which can be on the classpath or as a module. It's not
clear to me that we need all 8 combinations. It's probably sufficient to have a
reasonable subset.
An idea for possible future expansion is to mix automatic modules with named
modules. I'm not entirely sure how to do that. Perhaps one way is to have
module-info files for all the components, and then create variant jar files with
the module-info.class omitted. That way we'd have a modular jar, and then a
"plain" jar (without module-info.class) that'd be suitable for use as an
automatic module or to be put on the classpath. That'd be 3^3=27 combinations,
which I certainly think is overkill.
In any case, for this initial changeset, I think it's sufficient to test a few
combinations of automatic modules vs. classpath. We can extend the cases to
include named modules later. Please make a recommendation for some set of
combinations and implement it, and then send it out for a final round of review.
Thanks.
s'marks
Re: 8152650: ModuleFinder.compose should accept varargs
> On May 20, 2016, at 1:57 PM, Alan Bateman wrote: > > > There are several patches that didn't go in with the last update, the change > to ModuleFinder.compose to use varargs was one of them. The other part to > this is removing the empty() method as it is redundant. The changes are > straight-forward: > http://cr.openjdk.java.net/~alanb/8152650/webrev/ This looks good to me. Mandy
RE: Compact profiles broken?
Hi Alan,
Look here:
import javax.xml.bind.DatatypeConverter;
public class Test {
public static void main(String... args) throws Exception {
DatatypeConverter.parseBoolean("true");
}
}
$ javac Test.java
Test.java:1: error: package javax.xml.bind does not exist
import javax.xml.bind.DatatypeConverter;
^
Test.java:5: error: cannot find symbol
DatatypeConverter.parseBoolean("true");
^
symbol: variable DatatypeConverter
location: class Test
2 errors
$ javac -source 8 -target 8 Test.java
warning: [options] bootstrap class path not set in conjunction with -source 1.8
1 warning
Of course it also passes with -release 8, but thats very clear, because it uses
ct.sym of older version.
Uwe
-
Uwe Schindler
uschind...@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
> -Original Message-
> From: jigsaw-dev [mailto:jigsaw-dev-boun...@openjdk.java.net] On Behalf
> Of Uwe Schindler
> Sent: Saturday, May 21, 2016 12:23 AM
> To: 'Alan Bateman' ; jigsaw-
> d...@openjdk.java.net
> Subject: RE: Compact profiles broken?
>
> Hi,
>
> > > FYI, this also explains why we did not see a compile failure about the
> > java.xml.bind issue (see previous mails)! This only failed at runtime,
> because
> > with Jigsaw, the java.xml.bind is not exposed by the root modules. But
> while
> > compiling it seems to see all those classes, as its compiled against
> > source/target 8 where the module system is ignored. So there is also room
> > for improvement! Alan?
> > >
> > I think that's right as when compiling with -source/-target 8 they you
> > are also specifying -bootclasspath (at least I assume you are) and so
> > those types should be visible.
>
> No, we don't. But they were still visible to javac. But that’s fixed already
> in
> our build (we removed the class reference and we changed the build in Java
> 9, see below)!
>
> Please note: I changed the Lucene/Solr build to now use "-release" if it
> detects Java 9 (very easy with Apache Ant). Elasticsearch is still
> investiagting
> how to do this with Gradle... :(
>
> https://issues.apache.org/jira/browse/LUCENE-7292
>
> Uwe
RE: Compact profiles broken?
Hi, > > FYI, this also explains why we did not see a compile failure about the > java.xml.bind issue (see previous mails)! This only failed at runtime, because > with Jigsaw, the java.xml.bind is not exposed by the root modules. But while > compiling it seems to see all those classes, as its compiled against > source/target 8 where the module system is ignored. So there is also room > for improvement! Alan? > > > I think that's right as when compiling with -source/-target 8 they you > are also specifying -bootclasspath (at least I assume you are) and so > those types should be visible. No, we don't. But they were still visible to javac. But that’s fixed already in our build (we removed the class reference and we changed the build in Java 9, see below)! Please note: I changed the Lucene/Solr build to now use "-release" if it detects Java 9 (very easy with Apache Ant). Elasticsearch is still investiagting how to do this with Gradle... :( https://issues.apache.org/jira/browse/LUCENE-7292 Uwe
Re: Compact profiles broken?
On 19/05/2016 13:08, Uwe Schindler wrote: : FYI, this also explains why we did not see a compile failure about the java.xml.bind issue (see previous mails)! This only failed at runtime, because with Jigsaw, the java.xml.bind is not exposed by the root modules. But while compiling it seems to see all those classes, as its compiled against source/target 8 where the module system is ignored. So there is also room for improvement! Alan? I think that's right as when compiling with -source/-target 8 they you are also specifying -bootclasspath (at least I assume you are) and so those types should be visible. -Alan.
Re: RFR 8156497: Add jar tool support for Multi-release modular JARs
On 20/05/2016 16:55, Chris Hegarty wrote: : http://cr.openjdk.java.net/~chegar/8156497.00/ Note: while there are some new test scenarios added, which give reasonable coverage, further tests will be added later. Steve has some additional jar tools support coming for easier creation of MRJARS. The checks looks right. In checkModuleInfos then I assume you can use findFirst instead of collecting the module-infos into a set. The comment in computeHashes is confusing - too many "as" :-) You might want to re-read the error messages to see if they can be improved. For example "Invalid versioned module-info.class ..." might be better as "module-info.class in versioned section ...". Another one is "Unexpected versioned module-info.class without root module-info.class" where it might be clearer to say "module-info.class found in versions section of JAR file, no module-info.class found in root directory" or something like that. Looks like the copyright headers have the Classpath exception so now might be a good time to fix those. -Alan
8152650: ModuleFinder.compose should accept varargs
There are several patches that didn't go in with the last update, the change to ModuleFinder.compose to use varargs was one of them. The other part to this is removing the empty() method as it is redundant. The changes are straight-forward: http://cr.openjdk.java.net/~alanb/8152650/webrev/ I dropped the changes to the langtools repo as Jon/Vicente have a patch in flight that drops empty(). Also the last round of jdeps updates removed the use of empty(). -Alan.
Re: Weird use case: compiling against dummy sun.misc.* class
On 20/05/2016 18:02, Aleksey Shipilev wrote: : -Xmodule:jdk.unsupported definitely works, thanks! What are the drawbacks for enabling this? I can easily enable that globally for the entire project, but that would compile all project files as if in the jdk.unsupported module? I guess that does not matter if I don't produce/deploy the module as the build artifact. It would compile everyone as if part of this module. That module currently exports all its packages so there isn't a risk that you successfully compile with a reference to a type in a non-exported package. Also are you compiling jdk.internal.annotations.Contended too? That is in java.base. (Aside: by reading the javac source code, I discovered -XDnoModules, which also works, but hidden, unsupported, fragile, and scary) Jon has it on the list to remove, see JDK-8152785. -Alan
Re: Weird use case: compiling against dummy sun.misc.* class
On 05/20/2016 07:13 PM, Alan Bateman wrote: > On 20/05/2016 16:55, Aleksey Shipilev wrote: > jcstress is boot loader so I assume multi-release JARs are out of the > question? Yes, a simple workaround would be better than doing MR JARs at this point. We have >50 Mb JARs as it is, MR JARs would be at least twice as that if we ship JDK <=8 and JDK >=9 classes in the same package. > So what is the javac command that you are using? If you are compiling > sun/misc/Contented.java then I would expect to see `javac > -Xmodule:jdk.unsupported` so that it is compiled "as if" part of the > jdk.unsupported module. -Xmodule:jdk.unsupported definitely works, thanks! What are the drawbacks for enabling this? I can easily enable that globally for the entire project, but that would compile all project files as if in the jdk.unsupported module? I guess that does not matter if I don't produce/deploy the module as the build artifact. (Aside: by reading the javac source code, I discovered -XDnoModules, which also works, but hidden, unsupported, fragile, and scary) Thanks, -Aleksey
Re: RFR 8156497: Add jar tool support for Multi-release modular JARs
> On 20 May 2016, at 17:01, Jonathan Gibbons > wrote: > > It would be good if (eventually) there were test cases involving the use > of MRM jars, both via the JarFile API and jar-fs file system, perhaps > extending to the use of MRM jars in javac as well. I agree. Just to note, the updated test in the webrev is already exercising the JarFile API, indirectly. Support for Modular JARs on the module path uses the JarFile constructor that accepts a JarFile.Release value when opening. The test exercises this when it verifies the newly created, or updated, JAR file by running the module’s entry point. But I agree, more test scenarios will need to be added. -Chris. > -- Jon > > On 05/20/2016 08:55 AM, Chris Hegarty wrote: >> What do you get if you mix JEP 261 [1] with JEP 238 [2]? >> A Multi-release modular JAR. >> >> This issue proposes to add support to the jar tool for creating and >> updating modular JAR files with an optional module-info.class in the >> versioned section. >> >> MRJARs are intended to target multiple releases of the Java platform, so >> it seems reasonable for a Multi-release modular JAR ( MRMJAR ) to be >> able to declare a different set of dependencies on modules that are part >> of the Java platform. The reasoning here is that these are >> implementation details rather than parts of a module's API surface, and >> that one may well want to change them as the JDK itself evolves. The >> runtime already has support for loading from the versioned section. >> >> Specifically, a versioned module descriptor must be identical to the >> root module descriptor, with two exceptions: >> >> - A versioned descriptor can have different non-public `requires` >> clauses of platform ( `java.*` and `jdk.*` ) modules, and >> >> - A versioned descriptor can have different `uses` clauses, even of >> service types defined outside of `java.*` and `jdk.*` modules. >> >> http://cr.openjdk.java.net/~chegar/8156497.00/ >> >> Note: while there are some new test scenarios added, which give >> reasonable coverage, further tests will be added later. Steve has some >> additional jar tools support coming for easier creation of MRJARS. >> >> -Chris. >> >> [1] http://openjdk.java.net/jeps/261 >> [2] http://openjdk.java.net/jeps/238 >
Re: RFR 8156497: Add jar tool support for Multi-release modular JARs
On 20 May 2016, at 17:10, Alexandre (Shura) Iline wrote: > > Chris, > > Does the test need to use @modules for jdk.jartool and java.compiler? Yes, of course. Added, and updated in-place. http://cr.openjdk.java.net/~chegar/8156497.00/ -Chris. > > Shura > >> On May 20, 2016, at 8:55 AM, Chris Hegarty wrote: >> >> What do you get if you mix JEP 261 [1] with JEP 238 [2]? >> A Multi-release modular JAR. >> >> This issue proposes to add support to the jar tool for creating and >> updating modular JAR files with an optional module-info.class in the >> versioned section. >> >> MRJARs are intended to target multiple releases of the Java platform, so >> it seems reasonable for a Multi-release modular JAR ( MRMJAR ) to be >> able to declare a different set of dependencies on modules that are part >> of the Java platform. The reasoning here is that these are >> implementation details rather than parts of a module's API surface, and >> that one may well want to change them as the JDK itself evolves. The >> runtime already has support for loading from the versioned section. >> >> Specifically, a versioned module descriptor must be identical to the >> root module descriptor, with two exceptions: >> >> - A versioned descriptor can have different non-public `requires` >> clauses of platform ( `java.*` and `jdk.*` ) modules, and >> >> - A versioned descriptor can have different `uses` clauses, even of >> service types defined outside of `java.*` and `jdk.*` modules. >> >> http://cr.openjdk.java.net/~chegar/8156497.00/ >> >> Note: while there are some new test scenarios added, which give >> reasonable coverage, further tests will be added later. Steve has some >> additional jar tools support coming for easier creation of MRJARS. >> >> -Chris. >> >> [1] http://openjdk.java.net/jeps/261 >> [2] http://openjdk.java.net/jeps/238 >
Re: Weird use case: compiling against dummy sun.misc.* class
On 5/20/2016 9:13 AM, Alex Buckley wrote: If you have your own sun.misc.FOO on the classpath, then you DON'T want a module exporting sun.misc to be readable, because that module is where javac and the runtime will get sun.misc.* types from. So, no need to add readability of the jdk.unsupported module which is not read by default on JDK 9b118. I (sort of) take that back -- jdk.unsupported will be read at runtime since it exports at least one package without qualification, but I seem to recall something about not making it readable by default at compile time. Alex
Re: Weird use case: compiling against dummy sun.misc.* class
If you have your own sun.misc.FOO on the classpath, then you DON'T want a module exporting sun.misc to be readable, because that module is where javac and the runtime will get sun.misc.* types from. So, no need to add readability of the jdk.unsupported module which is not read by default on JDK 9b118. javac "should" be observing your sun.misc.Contended type out of the box -- there may be a javac bug or some other problem with how javac is invoked, please send more details. I assume that the jdk.internal.annotations package exists in java.base so at run time you'll need to -Xpatch your jdk.internal.annotations.Contended type into java.base. Alex On 5/20/2016 8:55 AM, Aleksey Shipilev wrote: Hi, I have a weird use case in jcstress around @Contended. In order to support both JDK 8 and JDK 9 we build against *our own* sun.misc.Contended and jdk.internal.annotations.Contended. This works arguably well for both compiling and running with both JDK 8 and JDK 9: the real annotation gets picked up from the JDK. However, since JDK 9b118 we cannot build anymore, because: LongResult1_jcstress.java:[3,13] error: Contended is not visible because package sun.misc is not visible ...even though sun/misc/Contended.java is right here in my source tree. Is there a magic incantation to make javac proceed? The usual tricks did not help (probably because the target class is not in jdk.unsupported, but rather in the source tree itself): -addmods jdk.unsupported -XaddExports:jdk.unsupported/sun.misc=ALL-UNNAMED -XaddReads:jdk.unsupported=ALL-UNNAMED Thanks, -Aleksey
Re: Weird use case: compiling against dummy sun.misc.* class
On 20/05/2016 16:55, Aleksey Shipilev wrote: Hi, I have a weird use case in jcstress around @Contended. In order to support both JDK 8 and JDK 9 we build against *our own* sun.misc.Contended and jdk.internal.annotations.Contended. This works arguably well for both compiling and running with both JDK 8 and JDK 9: the real annotation gets picked up from the JDK. However, since JDK 9b118 we cannot build anymore, because: LongResult1_jcstress.java:[3,13] error: Contended is not visible because package sun.misc is not visible ...even though sun/misc/Contended.java is right here in my source tree. Is there a magic incantation to make javac proceed? The usual tricks did not help (probably because the target class is not in jdk.unsupported, but rather in the source tree itself): -addmods jdk.unsupported -XaddExports:jdk.unsupported/sun.misc=ALL-UNNAMED -XaddReads:jdk.unsupported=ALL-UNNAMED jcstress is boot loader so I assume multi-release JARs are out of the question? So what is the javac command that you are using? If you are compiling sun/misc/Contented.java then I would expect to see `javac -Xmodule:jdk.unsupported` so that it is compiled "as if" part of the jdk.unsupported module. -Alan
Re: RFR 8156497: Add jar tool support for Multi-release modular JARs
Chris, Does the test need to use @modules for jdk.jartool and java.compiler? Shura > On May 20, 2016, at 8:55 AM, Chris Hegarty wrote: > > What do you get if you mix JEP 261 [1] with JEP 238 [2]? > A Multi-release modular JAR. > > This issue proposes to add support to the jar tool for creating and > updating modular JAR files with an optional module-info.class in the > versioned section. > > MRJARs are intended to target multiple releases of the Java platform, so > it seems reasonable for a Multi-release modular JAR ( MRMJAR ) to be > able to declare a different set of dependencies on modules that are part > of the Java platform. The reasoning here is that these are > implementation details rather than parts of a module's API surface, and > that one may well want to change them as the JDK itself evolves. The > runtime already has support for loading from the versioned section. > > Specifically, a versioned module descriptor must be identical to the > root module descriptor, with two exceptions: > > - A versioned descriptor can have different non-public `requires` > clauses of platform ( `java.*` and `jdk.*` ) modules, and > > - A versioned descriptor can have different `uses` clauses, even of > service types defined outside of `java.*` and `jdk.*` modules. > > http://cr.openjdk.java.net/~chegar/8156497.00/ > > Note: while there are some new test scenarios added, which give > reasonable coverage, further tests will be added later. Steve has some > additional jar tools support coming for easier creation of MRJARS. > > -Chris. > > [1] http://openjdk.java.net/jeps/261 > [2] http://openjdk.java.net/jeps/238
Re: RFR 8156497: Add jar tool support for Multi-release modular JARs
It would be good if (eventually) there were test cases involving the use of MRM jars, both via the JarFile API and jar-fs file system, perhaps extending to the use of MRM jars in javac as well. -- Jon On 05/20/2016 08:55 AM, Chris Hegarty wrote: What do you get if you mix JEP 261 [1] with JEP 238 [2]? A Multi-release modular JAR. This issue proposes to add support to the jar tool for creating and updating modular JAR files with an optional module-info.class in the versioned section. MRJARs are intended to target multiple releases of the Java platform, so it seems reasonable for a Multi-release modular JAR ( MRMJAR ) to be able to declare a different set of dependencies on modules that are part of the Java platform. The reasoning here is that these are implementation details rather than parts of a module's API surface, and that one may well want to change them as the JDK itself evolves. The runtime already has support for loading from the versioned section. Specifically, a versioned module descriptor must be identical to the root module descriptor, with two exceptions: - A versioned descriptor can have different non-public `requires` clauses of platform ( `java.*` and `jdk.*` ) modules, and - A versioned descriptor can have different `uses` clauses, even of service types defined outside of `java.*` and `jdk.*` modules. http://cr.openjdk.java.net/~chegar/8156497.00/ Note: while there are some new test scenarios added, which give reasonable coverage, further tests will be added later. Steve has some additional jar tools support coming for easier creation of MRJARS. -Chris. [1] http://openjdk.java.net/jeps/261 [2] http://openjdk.java.net/jeps/238
RFR 8156497: Add jar tool support for Multi-release modular JARs
What do you get if you mix JEP 261 [1] with JEP 238 [2]? A Multi-release modular JAR. This issue proposes to add support to the jar tool for creating and updating modular JAR files with an optional module-info.class in the versioned section. MRJARs are intended to target multiple releases of the Java platform, so it seems reasonable for a Multi-release modular JAR ( MRMJAR ) to be able to declare a different set of dependencies on modules that are part of the Java platform. The reasoning here is that these are implementation details rather than parts of a module's API surface, and that one may well want to change them as the JDK itself evolves. The runtime already has support for loading from the versioned section. Specifically, a versioned module descriptor must be identical to the root module descriptor, with two exceptions: - A versioned descriptor can have different non-public `requires` clauses of platform ( `java.*` and `jdk.*` ) modules, and - A versioned descriptor can have different `uses` clauses, even of service types defined outside of `java.*` and `jdk.*` modules. http://cr.openjdk.java.net/~chegar/8156497.00/ Note: while there are some new test scenarios added, which give reasonable coverage, further tests will be added later. Steve has some additional jar tools support coming for easier creation of MRJARS. -Chris. [1] http://openjdk.java.net/jeps/261 [2] http://openjdk.java.net/jeps/238
Weird use case: compiling against dummy sun.misc.* class
Hi, I have a weird use case in jcstress around @Contended. In order to support both JDK 8 and JDK 9 we build against *our own* sun.misc.Contended and jdk.internal.annotations.Contended. This works arguably well for both compiling and running with both JDK 8 and JDK 9: the real annotation gets picked up from the JDK. However, since JDK 9b118 we cannot build anymore, because: LongResult1_jcstress.java:[3,13] error: Contended is not visible because package sun.misc is not visible ...even though sun/misc/Contended.java is right here in my source tree. Is there a magic incantation to make javac proceed? The usual tricks did not help (probably because the target class is not in jdk.unsupported, but rather in the source tree itself): -addmods jdk.unsupported -XaddExports:jdk.unsupported/sun.misc=ALL-UNNAMED -XaddReads:jdk.unsupported=ALL-UNNAMED Thanks, -Aleksey
RE: Mutable modules
I was on a project using OSGi where some of the applications relied on removing
and replacing modules.
Generally they replaced applications modules at the top with no dependencies
into the module.
For example, in one application that dealt with RFID, it was common for new
RFID clients to be introduced or modifications required to existing RFID
clients (the technology was changing very quickly).
The handling of these clients was done with modules that could be swapped in as
necessary.
There were well defined interfaces with various implementations for the
different clients.
-Original Message-
From: Neil Bartlett [mailto:njbartl...@gmail.com]
Sent: Friday, May 20, 2016 11:20 AM
To: Alan Bateman
Cc: jigsaw-dev
Subject: Re: Mutable modules
> On 20 May 2016, at 15:12, Alan Bateman wrote:
>
> On 18/05/2016 22:47, David M. Lloyd wrote:
>>
>> I mean in *our* current concept of a module, we can add/remove/modify the
>> contents of a module (its "class path") at run time. It is up to the user
>> to ensure that doing so makes sense.
> I don't think I can relate to the use case. As you probably know then ZIP
> files have historically had their central directory mapped into memory.
> Removing or replacing a file that is memory mapped will likely lead to
> processes accessing the mapped file to crash (SIGBUS usually). So if users
> are really doing such hairy things they would need a lot of insight into what
> is running and whether the file is opened before taking this risk.
>
Mutable module graphs have an 18 year history in OSGi, so they are certainly
practical, of course with certain limitations. The use cases are to allow
runtime installation or upgrade of modules without shutting down an entire
application. Such capabilities proved indispensable in OSGi’s original market
of home gateways, and now in the IoT world.
They key point is that it is a cooperative mechanism. When a module is
uninstalled in OSGi, a callback is sent to the module in question, and it is
then responsible for safely cleaning up anything that it was doing. If it does
so properly then the module, along with all of its loaded objects, their
classes and the classloader, become available for garbage collection. When a
module is updated, a new classloader is created for the new version. Therefore
if a module fails to implement correct clean-up and is updated many times, you
have a memory leak. This is inescapable… the JVM can no more force a module to
unload then it can force a thread to stop, and we all know the dire
consequences of calling Thread#stop().
In case of a dependency (module A imports a package from module B), when B is
uninstalled then module A must be refreshed. This involves calling the shutdown
callback on modules A and B, and then re-resolving A. That resolution will
likely fail as it will not be able to import the required packaged (unless they
are exported from somewhere else).
Obviously this model depends heavily on a 1-to-1 mapping of classloaders to
modules, in common with the JBoss Module System as described previously by
David Lloyd.
>> :
>>
>> Our modules each correspond to their own class loader: so far so good, we
>> can just have one Module per class loader. Problem is that we support
>> circularity, and also we support dependencies that go across module systems
>> with isolated namespaces (basically, our module loaders are a higher order
>> of the exact same concept of class loaders).
> If there are cyclic relationships between your modules then it will be
> problematic. Do you see much of this? If you've read Alex's JavaOne slides
> then you'll know that some of us like Kirk Knoernschild's book on Java
> Application Architecture and section "4.4 Cyclic Dependencies - the Death
> Knell" where he poses the question "Are Cycles Always Bad?". I don't want to
> say too much on this topic here as it is listed as an open issue on the JSR
> issues list.
>
OSGi permits cyclical dependencies, but such cycles can only be resolved
all-at-once. This is essentially why OSGi separates its resolution phase (i.e.
solving all dependencies) from its activation phase. Modules involved in a
cycle will not be GC’d until all of the modules in that cycle are uninstalled
or refreshed.
>>
>> Our modules support specifications including the content of the module
>> ("resource loaders") and the dependencies of the module. At run time, custom
>> ModuleLoader implementations can change the resource loader list and/or the
>> dependency list at any time, causing the module to be relinked on the spot;
>> the most useful aspect of this is the ability to incrementally deploy
>> applications which may include circular dependencies.
> Aside from cycles then what other use-cases do you have here? I read "change
> the resource loader list" to mean that the set of resources in the module
> changes, which is a bit weird if those resources are class files that have
> already bee
Re: Mutable modules
I had one last follow-up thought, see below: On 05/20/2016 10:20 AM, David M. Lloyd wrote: You can't, on one hand, define a universal namespace and syntax for modules and their versions in the JDK or establish hard constraints on layer and module graph structure, and on the other hand expect other module systems with differing existing constraints to unify on the JDK module system. You're basically cutting these systems off at the knees and forcing them to reinvent everything, unless you completely coincidentally have a system that already conforms to this structure (if so, you are either very fortunate or maybe starting off in a rather privileged position). To be clear, what I'm advocating for is to separate these constraints from *at least* the diagnostic benefits and (f we can resurrect the access control discussion) the security benefits of JDK modules at run time. This might take the form of much greater autonomy for Layers as I've proposed in the issues. But, it might also take the form of allowing non-Jigsaw-style code to (separately, independently) arbitrarily create and manage Modules at run time, completely separate from descriptors, linking behavior, etc. In other words, if I can establish a Class or Package in my class loader, while at the same time defining its module diagnostic information (name, version string, location string, etc.) and security information (what packages can access the nonpublic members of this package, preferably a (possibly wholly or partly weak) set that can be added to at run time), then suddenly Jigsaw-style module artifacts can coexist with other module systems without any drawbacks that I can see, and other existing module systems will then gain access to the fancy stack traces I like so much and any new access checking capabilities that are established. The changes I or other maintainers need to make become minimal because the constraints that Jigsaw imposes are no longer stumbling blocks. Whether or not this means I have to define a real Module becomes immaterial from an implementation standpoint. I hope these ideas are making some kind of sense. -- - DML
Re: Mutable modules
On 05/20/2016 09:12 AM, Alan Bateman wrote:
On 18/05/2016 22:47, David M. Lloyd wrote:
I mean in *our* current concept of a module, we can add/remove/modify
the contents of a module (its "class path") at run time. It is up to
the user to ensure that doing so makes sense.
I don't think I can relate to the use case. As you probably know then
ZIP files have historically had their central directory mapped into
memory. Removing or replacing a file that is memory mapped will likely
lead to processes accessing the mapped file to crash (SIGBUS usually).
So if users are really doing such hairy things they would need a lot of
insight into what is running and whether the file is opened before
taking this risk.
No, we don't support replacement of JAR files; more like we symbolically
remove the resources and add overlays. Internally, we generally explode
JAR files for various reasons. It has nothing to do with JAR files,
more to do with the logical presence and absence of class files and
resources.
Our modules each correspond to their own class loader: so far so good,
we can just have one Module per class loader. Problem is that we
support circularity, and also we support dependencies that go across
module systems with isolated namespaces (basically, our module loaders
are a higher order of the exact same concept of class loaders).
If there are cyclic relationships between your modules then it will be
problematic. Do you see much of this? If you've read Alex's JavaOne
slides then you'll know that some of us like Kirk Knoernschild's book on
Java Application Architecture and section "4.4 Cyclic Dependencies - the
Death Knell" where he poses the question "Are Cycles Always Bad?". I
don't want to say too much on this topic here as it is listed as an open
issue on the JSR issues list.
We use circular dependencies in both our static module layouts and also
in our dynamic deployment system. I don't think we have a clear path
out if we can't support them; it will certainly be a difficult situation.
Our modules support specifications including the content of the module
("resource loaders") and the dependencies of the module. At run time,
custom ModuleLoader implementations can change the resource loader
list and/or the dependency list at any time, causing the module to be
relinked on the spot; the most useful aspect of this is the ability to
incrementally deploy applications which may include circular
dependencies.
Aside from cycles then what other use-cases do you have here? I read
"change the resource loader list" to mean that the set of resources in
the module changes, which is a bit weird if those resources are class
files that have already been loaded. Maybe there is dynamic code
generation with class bytes generated to the file system or into
somewhere virtual? Or maybe these resources are something else, data
files? I'm just trying to understand what you mean as we are using
differently terminology.
Within our module concept, the "class path" is a sort of colloquial term
which refers to the series of resource loaders used to locate classes
and resources within each module. Functionally it's somewhat analogous
to the URLClassPath concept in the JDK, in the way that resources are
sought (i.e. by a linear search from start to end of the list).
This is separate from the module dependency list, which, when combined
with the resource loader list, is used to construct an index by path
(which is a superset of packages that includes not just classes but also
resources) which refers to dependencies and/or internal resources.
When the resource list is changed, all future lookups for classes and
resources will use the new index. If there are already classes loaded
from the previous list, and those classes are sufficiently incompatible
with the new code, obviously this will result in errors; however, this
is usually not the case when (say) making incremental changes during
development. This is generally an edge case, but it is one which we
presently support.
Changing the dependency list has effects that are somewhat similar.
Existing loaded classes which have already linked against classes of the
previous dependency may malfunction when doing this. But in the hot
deployment situation, most of the time these changes are additive, so
the effect is generally to enable previously unlinked code to become
linkable. This could happen, for example, when deploying a JAR some
time after a first JAR was deployed, which resolves some missing
dependencies in the first JAR.
We also support delegating "fallback" class loading decisions to
outside suppliers for dynamic class loading behavior (this was done to
support a dynamic OSGi environment). The ongoing integrity of the
system is up to the party doing the relinking (the EE deployer or the
OSGi resolver); most of the time it can reason about what is "safe"
and what might cause system breakage (but still might be useful to do
anyway)
Re: Mutable modules
> On 20 May 2016, at 15:12, Alan Bateman wrote:
>
> On 18/05/2016 22:47, David M. Lloyd wrote:
>>
>> I mean in *our* current concept of a module, we can add/remove/modify the
>> contents of a module (its "class path") at run time. It is up to the user
>> to ensure that doing so makes sense.
> I don't think I can relate to the use case. As you probably know then ZIP
> files have historically had their central directory mapped into memory.
> Removing or replacing a file that is memory mapped will likely lead to
> processes accessing the mapped file to crash (SIGBUS usually). So if users
> are really doing such hairy things they would need a lot of insight into what
> is running and whether the file is opened before taking this risk.
>
Mutable module graphs have an 18 year history in OSGi, so they are certainly
practical, of course with certain limitations. The use cases are to allow
runtime installation or upgrade of modules without shutting down an entire
application. Such capabilities proved indispensable in OSGi’s original market
of home gateways, and now in the IoT world.
They key point is that it is a cooperative mechanism. When a module is
uninstalled in OSGi, a callback is sent to the module in question, and it is
then responsible for safely cleaning up anything that it was doing. If it does
so properly then the module, along with all of its loaded objects, their
classes and the classloader, become available for garbage collection. When a
module is updated, a new classloader is created for the new version. Therefore
if a module fails to implement correct clean-up and is updated many times, you
have a memory leak. This is inescapable… the JVM can no more force a module to
unload then it can force a thread to stop, and we all know the dire
consequences of calling Thread#stop().
In case of a dependency (module A imports a package from module B), when B is
uninstalled then module A must be refreshed. This involves calling the shutdown
callback on modules A and B, and then re-resolving A. That resolution will
likely fail as it will not be able to import the required packaged (unless they
are exported from somewhere else).
Obviously this model depends heavily on a 1-to-1 mapping of classloaders to
modules, in common with the JBoss Module System as described previously by
David Lloyd.
>> :
>>
>> Our modules each correspond to their own class loader: so far so good, we
>> can just have one Module per class loader. Problem is that we support
>> circularity, and also we support dependencies that go across module systems
>> with isolated namespaces (basically, our module loaders are a higher order
>> of the exact same concept of class loaders).
> If there are cyclic relationships between your modules then it will be
> problematic. Do you see much of this? If you've read Alex's JavaOne slides
> then you'll know that some of us like Kirk Knoernschild's book on Java
> Application Architecture and section "4.4 Cyclic Dependencies - the Death
> Knell" where he poses the question "Are Cycles Always Bad?". I don't want to
> say too much on this topic here as it is listed as an open issue on the JSR
> issues list.
>
OSGi permits cyclical dependencies, but such cycles can only be resolved
all-at-once. This is essentially why OSGi separates its resolution phase (i.e.
solving all dependencies) from its activation phase. Modules involved in a
cycle will not be GC’d until all of the modules in that cycle are uninstalled
or refreshed.
>>
>> Our modules support specifications including the content of the module
>> ("resource loaders") and the dependencies of the module. At run time, custom
>> ModuleLoader implementations can change the resource loader list and/or the
>> dependency list at any time, causing the module to be relinked on the spot;
>> the most useful aspect of this is the ability to incrementally deploy
>> applications which may include circular dependencies.
> Aside from cycles then what other use-cases do you have here? I read "change
> the resource loader list" to mean that the set of resources in the module
> changes, which is a bit weird if those resources are class files that have
> already been loaded. Maybe there is dynamic code generation with class bytes
> generated to the file system or into somewhere virtual? Or maybe these
> resources are something else, data files? I'm just trying to understand what
> you mean as we are using differently terminology.
>
>> We also support delegating "fallback" class loading decisions to outside
>> suppliers for dynamic class loading behavior (this was done to support a
>> dynamic OSGi environment). The ongoing integrity of the system is up to the
>> party doing the relinking (the EE deployer or the OSGi resolver); most of
>> the time it can reason about what is "safe" and what might cause system
>> breakage (but still might be useful to do anyway). These are the features
>> we can't seem to
Re: Mutable modules
On 18/05/2016 22:47, David M. Lloyd wrote:
I mean in *our* current concept of a module, we can add/remove/modify
the contents of a module (its "class path") at run time. It is up to
the user to ensure that doing so makes sense.
I don't think I can relate to the use case. As you probably know then
ZIP files have historically had their central directory mapped into
memory. Removing or replacing a file that is memory mapped will likely
lead to processes accessing the mapped file to crash (SIGBUS usually).
So if users are really doing such hairy things they would need a lot of
insight into what is running and whether the file is opened before
taking this risk.
:
Our modules each correspond to their own class loader: so far so good,
we can just have one Module per class loader. Problem is that we
support circularity, and also we support dependencies that go across
module systems with isolated namespaces (basically, our module loaders
are a higher order of the exact same concept of class loaders).
If there are cyclic relationships between your modules then it will be
problematic. Do you see much of this? If you've read Alex's JavaOne
slides then you'll know that some of us like Kirk Knoernschild's book on
Java Application Architecture and section "4.4 Cyclic Dependencies - the
Death Knell" where he poses the question "Are Cycles Always Bad?". I
don't want to say too much on this topic here as it is listed as an open
issue on the JSR issues list.
Our modules support specifications including the content of the module
("resource loaders") and the dependencies of the module. At run time,
custom ModuleLoader implementations can change the resource loader
list and/or the dependency list at any time, causing the module to be
relinked on the spot; the most useful aspect of this is the ability to
incrementally deploy applications which may include circular
dependencies.
Aside from cycles then what other use-cases do you have here? I read
"change the resource loader list" to mean that the set of resources in
the module changes, which is a bit weird if those resources are class
files that have already been loaded. Maybe there is dynamic code
generation with class bytes generated to the file system or into
somewhere virtual? Or maybe these resources are something else, data
files? I'm just trying to understand what you mean as we are using
differently terminology.
We also support delegating "fallback" class loading decisions to
outside suppliers for dynamic class loading behavior (this was done to
support a dynamic OSGi environment). The ongoing integrity of the
system is up to the party doing the relinking (the EE deployer or the
OSGi resolver); most of the time it can reason about what is "safe"
and what might cause system breakage (but still might be useful to do
anyway). These are the features we can't seem to support under
Jigsaw, architecturally speaking.
This sounds like class loader delegation to resolve types that are not
in the module.
Specifically this includes (but is not limited to) changing the
package set associated with a JDK module at run time, something that
this native code block makes impossible. Also the ability to
dynamically change module dependencies is an essential ingredient to
making this work.
Suppose that module m has package p and p.C has been loaded. Are you
saying that you can drop package p from the module?
As things currently stand in JDK 9 then packages may be added to modules
at runtime, the main use case is the dynamic proxy to a public interface
in a non-exported packages. So I can relate to adding packages for code
gen cases, I'm less sure about a module starting out as an XML API and
suddenly changing into a JDBC driver. Do you really mean the same module
instance?
In my view, architecturally speaking, most of the constraints imposed
by the core module framework should be layer policy. If the system's
core module layer wants to maintain strict, static integrity, name
constraints, version syntax and semantics, etc., that's fine, but why
should all modules everywhere be forced to the same constraints?
Using module names as an example, then it should be possible to develop
a module that is deployed on the application module path or instantiated
in a layer of modules that a container creates. The author of the module
(that chooses the name) isn't going to know in advance how the module is
deployed. I'm not even sure how such a module could be compiled or how
anyone could depend on it when the characters or format can vary like
this. I see there is an issue on the JSR issues list so I don't want to
say any more on this topic.
There is no way that existing containers and class loading
environments (other than, apparently, WebLogic) can conform to
Jigsaw's constraints without losing functionality (and I'm trying hard
to find ways to make it work). This is where most of my raised issues
are com
