RE: Java module graph png settings?
Hi Patrick, you might be interested to have a look at a small tool "depvis" which I wrote together with a few colleagues. See https://github.com/accso/java9-jigsaw-depvis Tested with b144 and GraphViz 2.38. DepVis does also create GraphViz DOT files for Jigsaw module relationships and takes their various dependencies and relationships into account. For an example (= the graph of JDK9 system modules) see https://github.com/accso/java9-jigsaw-depvis/raw/master/Sample-J9SystemModules.png Features (all configurable to de/activate): 1. It prints different graph edges for - requires relationships - requires transitive relationships (1-transitive) - requires mandated relationships - directed exports (i.e. exports to) relationships 2. It prints different graph nodes for different modules (explicit, open, automatic). 3. It can filter out modules based on their names (with blacklists, wildcards and whitelists) 4. Optional legend More ideas to come, see https://github.com/accso/java9-jigsaw-depvis/blob/master/README.md#todos-lop-backlog-ideas- Any feedback much appreciated and welcome! Cheers, Martin -Original Message- From: jigsaw-dev [mailto:jigsaw-dev-boun...@openjdk.java.net] On Behalf Of Mandy Chung Sent: Saturday, December 10, 2016 7:12 AM To: Patrick ReinhartCc: jigsaw-dev Subject: Re: Java module graph png settings? dot -Tpng Mandy > On Dec 9, 2016, at 4:58 PM, Patrick Reinhart wrote: > > Hi Mandy, > > Can you tell me what options with the Graphviz framework are to get the > actual PNG file out of the jdk.dot file? I tried to find this, but did not > find the actual place. > > -Patrick > > >> Am 08.12.2016 um 07:18 schrieb Mandy Chung : >> >> Hi Patrick >> >> Are you looking for: >> >> http://hg.openjdk.java.net/jdk9/dev/jdk/file/c9785b0f04fd/make/src/classes/build/tools/jigsaw/GenGraphs.java >> >> Mandy >> >>> On Dec 7, 2016, at 1:17 PM, Patrick Reinhart wrote: >>> >>> Can anyone point me to the place, where the java module dependencies are >>> created? I would like to do something similar and I curious how it’s being >>> done exactly… >>> >>> Thanks very much >>> >>> -Patrick >> >
#VersionsInModuleNames too restrictive (cuts of too much)
Hi all, the new #VersionsInModuleNames seems to be active in JDK9 b136 already. Switching to b136 breaks the build in one of our examples of our Java9 Jigsaw example suite, cf Github: https://github.com/accso/java9-jigsaw-examples/tree/master/jigsaw-examples/e xample_automatic-module-logging In this example we are using the automatic module which is taken from the JAR file slf4j-jdk14-1.7.12.jar . This JAR had been taken from Maven central, cf https://search.maven.org/#artifactdetails%7Corg.slf4j%7Cslf4j-jdk14%7C1.7.21 %7Cjar Its Maven POM file says: org.slf4j slf4j-jdk14 1.7.21 Unfortunately, the version name which is automatically detected and taken from the Jigsaw module name is "14-1.7.21". So instead of calling the main class (as done with b127 ff) as follows: $JAVA_HOME/bin/java --module-path mlib:amlib --add-modules slf4j.jdk14 -m modmain/pkgmain.Main we now have to run the thing with: $JAVA_HOME/bin/java --module-path mlib:amlib --add-modules slf4j.jdk -m modmain/pkgmain.Main This seems like an implementation bug by cutting off too much of the suffix string. Please fix the implementation and also please consider to skip this feature completely: Frankly, the whole idea of trying to smart detecting and removals of version strings feels totally wrong to me. I understand the concerns not to let version numbering slip into module names. But this should not be done via some magic implementation but via good arguments , use case examples, patterns, good namin conventions. Note: If someone really wants to put in his/her versions into the module names, he/she always will find a way (like by using the number in the middle or as a prefix or ...). How could you really prevent that (and why would you)?. Thanks in advance, Martin
Follow-up question on new proposals for #ResourceEncapsulation & #ClassFilesAsResources
Hi all, I have a follow-up question on the new proposal on #ResourceEncapsulation, which states: > The effective package name of a resource named by the string > `"/foo/bar/baz"`, e.g., is 'foo.bar'. The idea of using resource paths as "effective package names" definitely makes sense to me. Sounds like a pragmatic way of getting this important requirement finally done. Great! But with the new module system, classes on the unnamed package are no longer allowed. The compiler rejects them with "error: unnamed package is not allowed in named modules". I am wondering what's going to happen with resources on the "unnamed path", so to say. It's not uncommon that resources like a "log4j.properties" are put and searched on the top-level path - i.e. the equivalent of the unnamed (class) package. What will happen to that at compile time (probably nothing?), at packaging time (jar does complain?) and at runtime? Thanks in advance, cheers, Martin -Original Message- From: jigsaw-dev [mailto:jigsaw-dev-boun...@openjdk.java.net] On Behalf Of Mark Reinhold Sent: Monday, September 12, 2016 5:20 PM To: jigsaw-dev@openjdk.java.net Subject: More proposals for open JPMS issues FYI, I've just posted new or revised proposals for some of the open issues in the JPMS specification: #ReflectiveAccessToNonExportedTypes & #AwkwardStrongEncapsulation http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-September/0003 90.html #AddExportsInManifest http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-September/0003 91.html #ResourceEncapsulation & #ClassFilesAsResources http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-September/0003 92.html #VersionsInModuleNames http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-September/0003 93.html #ClassLoaderNames http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-September/0003 94.html #ServiceLoaderEnhancements http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-September/0003 95.html Links to the proposals are also available in the issue summary [1]. The first proposal, for #ReflectiveAccessToNonExportedTypes and #AwkwardStrongEncapsulation, is likely to be of the broadest interest since it makes significant changes to the syntax and semantics of module declarations. The latter issue is a new issue, reported by Martin Buchholz and Aleksey Shipilev, and is summarized thus: #AwkwardStrongEncapsulation --- A non-public element of an exported package can still be accessed via the `AccessibleObject::setAccessible` method of the core reflection API. The only way to strongly encapsulate such an element is to move it to a non-exported package. This makes it awkward, at best, to encapsulate the internals of a package that defines a public API. The present design suffers this limitation due to an earlier compromise made to accommodate reflective access by frameworks, but experience has now shown that to be a problematic approach. It would be unfortunate indeed to bake this limitation into the module system for all time: It would make it much more difficult for developers to strongly encapsulate the internals of their own modules, yet enabling such encapsulation is one of the primary goals of this project. Thus this new proposal, which introduces the concepts of weak modules and private exports and removes the previously-proposed notion of dynamic exports. This has taken some time to work out but, in the end, appears to achieve a better balance of usability, ease of migration, and expressive power. Comments and discussion are welcome here on jigsaw-dev but, as usual, the best way to ensure that the EG sees any specific comment is to send it to the EG's "suggestion box" list, jpms-spec-comments [2]. If you comment on one of these proposals, via any channel, please include its hashtag in the subject line of your message to simplify tracking. - Mark [1] http://openjdk.java.net/projects/jigsaw/spec/issues/ [2] http://mail.openjdk.java.net/mailman/listinfo/jpms-spec-comments
RE: A few questions on layers ... and a compile problem with split-package caused by automatic module
Hi Alan, thanks for your answers. Sorry for the late answer but I meanwhile uploaded the example to Github: https://github.com/accso/java9-jigsaw-examples/tree/master/jigsaw-examples/e xample_layer-hierarchy mod.main: LayerBuilder does the startup of Jigsaw layers: https://github.com/accso/java9-jigsaw-examples/blob/master/jigsaw-examples/e xample_layer-hierarchy/src/mod.main/pkgmain/LayerBuilder.java ModuleCaller does the reflective calls to other modules: https://github.com/accso/java9-jigsaw-examples/blob/master/jigsaw-examples/e xample_layer-hierarchy/src/mod.main/pkgmain/ModuleCaller.java mod.x_bottom/middle/top have been mentioned already. The other modules are similar to mod.x*: mod.y* is for class derivation across module/layer boundaries. mod.z* is an example for delegation across module/layer boundaries. mod.layer for my own tree structure of layers: https://github.com/accso/java9-jigsaw-examples/tree/master/jigsaw-examples/e xample_layer-hierarchy/src/mod.layer It is constructed from a JSON file and each layer node in the tree has a name, keeps references to parent / child nodes in the tree.. and also a reference to the Jigsaw layer. PS: Do I understand correctly, that a Jigsaw layer can see its parent layer, but there is no API to retrieve its children layers? If not, is there any way to globally retrieve "all layers" for iteration? I think, that I once saw a similar question on this mailing list but cannot find the mail thread right now... > > Question 1) Is there any way to give a layer a name or an ID? My > > current workaround is to put my layers to a map which do the name <-> > > layer-instance mapping which is not nice. Please consider to allow layer names, thanks. > Layers don't have names. It's come up once or twice in the context of diagnosability as it's potentially useful to have in stack traces. I think TBD as to whether to do this or not, esp. given the number of fields that are now going into the stack trace element. Yes, please consider to add that functionality. Will also help in regression tests (as Layer.toString() will print a different hashCode every time, so result containing the layer info will differ). > > What's the visibility of modules (i.e. their classes) along the layer > > hierarchy / relationships? I had expected that a module (i.e. its > > classes) can only see modules (i.e. classes) in the same layer or in > > its parent or in any of its parent layers. > > I have tried this with a class hierarchy where a class in "bottom" is > > derived from a super class in "middle" which is again derived from a > > super class in "top". For that readability is needed from bottom -> middle -> top. > > Works as expected. > When using the Layer defineModulesWithXXX methods then the class loader delegation supports the readability graph. So from a module then you will be able to "see" any of the types in the packages exported by the modules that is > reads. In addition, I note in your example that you've specified the system class loader as the parent class loader. This means that you can "see" types that are visible via the system class loader too (assuming no overlapping packages in the graph of modules). Ok, thanks. > > On the other hand, my starter module mod.main is in the topmost boot layer. > > Via reflection, it can see and call anything from there in any module (i.e. > > also classes in "lower layers", i.e. in top, middle and bottom). > Are you using Class.forName and specifying the class loader for top, middle or bottom? No, I am not using Class.forName. Instead I am using layer.findLoader. See line #64 here: https://github.com/accso/java9-jigsaw-examples/blob/master/jigsaw-examples/e xample_layer-hierarchy/src/mod.main/pkgmain/ModuleCaller.java > Alternatively maybe one of the types in modules defined to these layers is leaking back to code in mod.main. Hm, no. There is no relationship from any of the mod.x* modules to mod.main, see here https://github.com/accso/java9-jigsaw-examples/blob/master/jigsaw-examples/e xample_layer-hierarchy/src/mod.main/module-info.java All is done via reflection. > > > > If readability and accessibility are given either statically in module-infos > > or dynamically at runtime, then are there any constraints of what can be > > seen / called? > If you are using the 3-arg Class.forName then this allows you to get a > reference to any type in any module. > > Question 3) > > Is there any way to add / delete modules from a given configuration? For > > now I used the 2 code lines above (***) to use the parent layer's > > configuration when creating a new configuration for a new layer. > > I have not found any API to add / delete modules "on the fly". Did I miss > > something? > No, that is different design. I think I am missing your point here... Do you mean "No, that would a different design and is (currently) not supported/available". Or do you mean: "Well, yes, there is a different API for that". (If the
RE: Question on returning instances of a (not exported) derived class
Hi Peter, thanks for your explanations. I do now understand the behaviour of case [1] ..., thanks! About case [2] you wrote * Are you sure you didn't have the toString() method overridden in InternalData at time of compilation? Yes, I am sure. I agree, this should compile. At least this compiler error message is confusing. About case [3] you wrote * invokevirtual #X // Method pkginternal/InternalData.toString:()Ljava/lang/String; * ...because the compiler notices the static type of the .toString() target is InternalData which declares the toString() method (and overrides Object::toString method). So InternalData should be accessible at runtime for this invocation to succeed and at compile time for compilation to succeed. I am a bit confused here, as you mention toString() . But in case [3] getName() is not related to toString() - at least not that I can see. InternalData.getName() overwrites Data.getName() , both returning a hard-coded String as result value. So did you mean getName() instead of toString() ? I.e. did you mean: * ...because the compiler notices the static type of the ***.getName()*** target is InternalData which declares the ***getName()*** method (and overrides ***Data::getName*** method). So InternalData should be accessible at runtime for this invocation to succeed and at compile time for compilation to succeed. If you did not mean this, I would be lost here... ;-) Finally: * Note that the following should compile though (and run too): ((Data) myDataFactory.createInternalData2()).getName() Yep, indeed. Compiles and runs. Thanks again. Cheers, Martin From: Peter Levart [mailto:peter.lev...@gmail.com] Sent: Thursday, August 25, 2016 1:42 PM To: Martin Lehmann <martin.lehm...@gmx.de>; jigsaw-dev@openjdk.java.net Subject: Re: Question on returning instances of a (not exported) derived class Hi Martin, Let me try to explain why... On 08/25/2016 08:56 AM, Martin Lehmann wrote: package pkgmain; import pkgx.*; public class Main { public static void main(String[] args) { DataFactory myDataFactory = new DataFactory(); // all OK System.out.println("Factory.createData(): " + myDataFactory.createData().getName()); System.out.println("Factory.createInternalData1().toString(): " + myDataFactory.createInternalData1().toString()); System.out.println("Factory.createInternalData1(): " + myDataFactory.createInternalData1().getName()); ...these, I hope, are understandable. You are invoking the methods upon the exported type. // [1]*does* compile though return type of // 'DataFactory.createInternalData2()' is not visible here System.out.println("Factory.createInternalData2(): " + myDataFactory.createInternalData2()); "Factory.createInternalData2(): " + myDataFactory.createInternalData2() ...is (or was, until replaced with an invokedynamic which should behave the same) equivalent to calling: new StringBuilder().append("Factory.createInternalData2(): ").append(myDataFactory.createInternalData2()).toString() ...the 2nd append is StringBuilder::append(Object) method. Compiler knows that InternalData type will not be needed to invoke that method at runtime, so it doesn't require you to have access to that type at compile time either. // [2] does not compile. error: toString() in Object is // defined in an inaccessible class or interface // surprise! why is [1] OK while adding toString() is not? System.out.println("Factory.createInternalData2().toString(): " + myDataFactory.createInternalData2().toString()); I think this is to restrictive. If the compiler succeeded in compiling this code, It would compile the toString() invocation as: invokevirtual #4 // Method java/lang/Object.toString:()Ljava/lang/String; ...which doesn't need to access InternalData type at runtime. So the compile time error should not be there IMO. Are you sure you didn't have the toString() method overridden in InternalData at time of compilation? Maybe the compiler is intentionally over restrictive here so that this type of code change (adding overriding public method to the concealed class) can not break the compilation? The error message is confusing, at least: "toString() in Object is defined in an inaccessible class or interface". How can Object be inaccessible? // [3] does not compile. error: getName() in InternalData is // defined in an inaccessible class or interface // suprise! getName() is overloaded, available in exported class Data System.out.println("Factory.createInternalData2().getName(): " + myDataFactory.createInternalData2().getName()); } } This would compile down to: invokevirtual #X // Method pkginternal/InternalData.toString:()Ljava/lang/String; ...becau
Question on returning instances of a (not exported) derived class
Hi all, I have a question on exporting packages and its handling with derived classes. Any help appreciated! In a module mod.x one package "pkgx" is exported containing 2 classes Data and DataFactory. Another package "pkgxinternal" is not exported and contains the class InternalData which is derived from Data. When the DataFactory returns an instance of Data, another module mod.main can work with its return value, of course. When the DataFactory returns an instance of InternalData, then this works fine, *if* the instance is returned as type Data (i.e. of the exported package). But when the DataFactory returns an instance of InternalData (i.e. of the not-exported package), then something weird comes up: DataFactory does of coure compile. But class Main in module mod.main does not: [1] This works fine (i.e. compiles & runs): System.out.println("Factory.createInternalData2(): " + myDataFactory.createInternalData2()); This should call toString (from Object, as neither Data nor InternalData have overloaded toString) [2] This does not compile. Compile error is "toString() in Object is defined in an inaccessible class or interface" System.out.println("Factory.createInternalData2(): " + myDataFactory.createInternalData2().toString()); The only difference between [1] and [2] is having added ".toString()" ! [3] This does also not compile. Compile error is "getName() in InternalData is defined in an inaccessible class or interface" System.out.println("Factory.createInternalData2(): " + myDataFactory.getName()); The method getName() in InternalData overloads the method in Data, which should be visible outside. Any explanation, why [1] is compiling but not [2] ... and why [3] is also not compiling? I have tested this with b132 (jdk1.9.0_ea-b132-x64_20160822_build5414). Thx in advance. Cheers, Martin # --- src/mod.x/module-info.java - module mod.x { exports pkgx; } # --- src/mod.x/pkgx/Data.java - package pkgx; public class Data { public String getName() { return "is Data"; } } # --- src/mod.x/pkgxinternal/InternalData.java - package pkginternal; public class InternalData extends pkgx.Data { @Override public String getName() { return "is InternalData"; } } # --- src/mod.x/pkgx/DataFactory.java package pkgx; public class DataFactory { // returns data which is exported in Module mod.x public Data createData() { return new Data(); } // returns data which is not exported from Module mod.x // (but as type of the exported super class, i.e. as type Data) public Data createInternalData1() { return new pkginternal.InternalData(); } // returns data which is not exported from Module mod.x // i.e. as type InternalData public InternalData createInternalData2() { return new pkginternal.InternalData(); } } # --- src/mod.main/module-info.java module mod.main { requires mod.x; } # --- src/mod.main/pkgmain/Main.java ---DOES NOT COMPILE BECAUSE OF [2] and [3] -- package pkgmain; import pkgx.*; public class Main { public static void main(String[] args) { DataFactory myDataFactory = new DataFactory(); // all OK System.out.println("Factory.createData(): " + myDataFactory.createData().getName()); System.out.println("Factory.createInternalData1().toString(): " + myDataFactory.createInternalData1().toString()); System.out.println("Factory.createInternalData1(): " + myDataFactory.createInternalData1().getName()); // [1]*does* compile though return type of // 'DataFactory.createInternalData2()' is not visible here System.out.println("Factory.createInternalData2(): " + myDataFactory.createInternalData2()); // [2] does not compile. error: toString() in Object is // defined in an inaccessible class or interface // surprise! why is [1] OK while adding toString() is not? System.out.println("Factory.createInternalData2().toString(): " + myDataFactory.createInternalData2().toString()); // [3] does not compile. error: getName() in InternalData is // defined in an inaccessible class or interface // suprise! getName() is overloaded, available in exported class Data System.out.println("Factory.createInternalData2().getName(): " + myDataFactory.createInternalData2().getName()); } }
RE: Avoiding same-package conflicts
Hi David, hi all, thanks, David, for your response. >> Full ACK. Bad practice. >I disagree, actually. I think that this is a completely needless and >artificial restriction that arose from implementation decisions, not from a >valid requirement. You have a point. I don't disagree ;-) >> So we really need disjunct classes in *all* libraries now? Not even, if the >> redundant packages are not even exported (right?). Would it work in the >> "old" classpath? > I think that judging by whether something would work in an old classpath is a > false test. I see your point, but: On the one hand: Migration will be a big issue. I followed the Monday JavaOne live streams talks (@Alan, @Alex: Thanks for the great talks!) on the topic and understood that stuff should work like before. Well, ambitious enough, but if this is indeed the goal, then "did it work in the old classpath?" is definitely something to look at. On the other hand - don't get me wrong: I personally prefer to have a clean and proper solution without (too much) of bad design *just* because backward compatibility. > The whole point of modules is to throw off the restrictions and problems of > the classpath, after all. I think a better test is, "does this make sense?", > and to use your slf4j example - it definitely makes sense, because: > ... Yep, agree. Btw, I meanwhile thought of a second use case which will *definitely* be needed. Assume to use a third-party library where something is wrong. Fix is not (yet) available, so you need to fix something locally. Up to now a common though bad(?) practice is to patch the library class(es) locally and put it the patched class(es) on the classpath *before* the third-party library (... as a workaround until the real fix is there). I doubt that I am the only one who ever had the need for such a workaround. If redundant packages are not longer allowed, then I (we?) need a replacement for such a scenario. (Or are I am now really expected to either encapsulate external stuff in different layers and/or to repackage external jar files to guarantee disjunct contents...?) Makes sense? Cheers, Martin -Original Message- From: jigsaw-dev [mailto:jigsaw-dev-boun...@openjdk.java.net] On Behalf Of David M. Lloyd Sent: Thursday, October 29, 2015 12:33 PM To: jigsaw-dev@openjdk.java.net Subject: Re: Avoiding same-package conflicts On 10/29/2015 05:59 AM, Martin Lehmann wrote: > Hi all, > > I stumbled across the same "issue". > >> that having multiple versions of the same library isn't a best practice to >> say the least. > Full ACK. Bad practice. I disagree, actually. I think that this is a completely needless and artificial restriction that arose from implementation decisions, not from a valid requirement. We have used our ability to ship multiple modules with the same package names to good effect on more than one occasion. As long as a given module doesn't link the conflicting packages, there is no real problem. Even hypothetical issues involving indirect linkage or passing objects back and forth have never really been a problem for us. > I might have an (artifical?) use case where this might lead to some extra > effort in migration, though: > Let's assume that two libraries D1 and D2 implement the same common API. For > some historical reason, both libraries *ship* the interfaces classes of that > common API. > This won't work (easily) without repackaging jar files (even if the common > API classes would not even differ). > > Example: > A --requires --> B, C > B --requires --> D1 > C --requires --> D2 > D1 and D2 both contain the same interfaces. Even if neither B nor C would > *not* "requires public" D1 / D2, this would not work. > > Too artifical? I actually thought of two logging implementations using & > shipping the common slf4j interface classes. > > So we really need disjunct classes in *all* libraries now? Not even, if the > redundant packages are not even exported (right?). Would it work in the "old" > classpath? I think that judging by whether something would work in an old classpath is a false test. The whole point of modules is to throw off the restrictions and problems of the classpath, after all. I think a better test is, "does this make sense?", and to use your slf4j example - it definitely makes sense, because: * Modules generally use slf4j as a private API to perform their own logging functions * Thus slf4j generally never leaks to a module's exported package set * Thus there's no actual harm in allowing this, and much benefit - especially if B and C (in your example) are not completely under your control (which is inevitable in any nontrivial system) -- - DML