Re: Current Project Status

2022-04-03 Thread Remi Forax
- Original Message -
> From: "Alan Bateman" 
> To: "Julian Waters" , "jigsaw-dev" 
> 
> Sent: Sunday, April 3, 2022 4:13:53 PM
> Subject: Re: Current Project Status

> On 03/04/2022 09:26, Julian Waters wrote:
>> Out of curiosity, is the Jigsaw project still active in the background
>> today, seeing that this mailing list is still up?
>>
> The features developed by this project were integrated in JDK 9. The
> mailing list lives on, it is used periodically for questions, bug
> reports, and related discussion.

The migration Java 8 -> 9, now Java 8 -> 11 has been slow,
even if major frameworks/libraries are now Java 11 compatible, there is still a 
long tail of applications in that process,
so i think it's a good idea to keep this list open.

> 
> -Alan

Rémi


Re: Annotation Dependencies and Requires Static Transitive

2021-06-03 Thread Remi Forax
- Mail original -
> De: "Anand Beh" 
> À: "jigsaw-dev" 
> Envoyé: Jeudi 3 Juin 2021 22:10:11
> Objet: Annotation Dependencies and Requires Static Transitive

> Hello,
> 
> The cache library Caffeine recently added a full module descriptor. It
> has no runtime dependencies, but it depends on metadata annotations
> from checker-qual and errorprone, for example @NotNull and
> @CanIgnoreReturnValue. The module looks like this:
> module com.github.benmanes.caffeine {
>  exports com.github.benmanes.caffeine.cache;
>  exports com.github.benmanes.caffeine.cache.stats;
> 
>  requires static transitive com.google.errorprone.annotations;
>  requires static transitive org.checkerframework.checker.qual;
> }
> 
> The annotations are not required at runtime, hence static. They're
> visibly placed on public methods and return types, so that API clients
> can benefit from them for the purposes of annotation-based null
> analysis, kotlin interop, etc. As the annotations are part of the API,
> they're marked transitive.
> 
> However, the "transitive" aspect imposes some requirements on users. I
> am wondering if there is a more correct way to declare these
> annotation dependencies than static transitive.
> 
> One user would like to avoid the presence of these annotations at
> compile-time. For reference, here's the relevant discussion:
> https://github.com/ben-manes/caffeine/issues/535
> 
> I'm not a maintainer of caffeine, though I was involved in its modularization.

Hi,
i've used just "requires static" without "transitive" for non null/nullable 
annotations in the past.

Thos annotations are consumed by tools, i don't think transitive is necessary.

> 
> Regards,
> Anand

regards,
Rémi


Re: Add reads to open module

2021-05-17 Thread forax
- Mail original -
> De: "Christian Beikov" 
> À: "Remi Forax" 
> Cc: "jigsaw-dev" 
> Envoyé: Lundi 17 Mai 2021 18:27:01
> Objet: Re: Add reads to open module

> Am 17.05.2021 um 18:10 schrieb Remi Forax:
>> - Mail original -
>>> De: "Christian Beikov" 
>>> À: "jigsaw-dev" 
>>> Envoyé: Lundi 17 Mai 2021 17:43:15
>>> Objet: Fwd: Add reads to open module
>>> Hi,
>>>
>>> I have a use case where I generate some code and that code refers to
>>> classes of some module B.
>>>
>>> When the module A (an open module), into which I define that class, has
>>> no read-edges to the module B, this fails saying that A does not read B.
>>>
>>> That's fine and totally understandable, so I tried to add the following
>>> code:
>>>
>>> moduleA.addReads("moduleB.pkg", moduleB)
>>>
>>> but this is not allowed as `addReads` only works if `this == callerModule`.
>>>
>>> My workaround is to define a class into module A that can then call this
>>> method.
>>>
>>> Is there a reason to this limitation? I mean the module is open anyway,
>>> so anyone can just define a class into it, so why not also allow calling
>>> addReads?
>> It's basic security [1], you can not see something you do not ask for.
>> By example, if your module as a security issue, you don't want an attacker 
>> to be
>> able to see all other open modules from your module.

> Right, but the module is open already, so I can do anything by injecting
> code into that module. It's more of a usabulity question I guess,
> because I'd rather just call `Module#addReads` directly instead of doing
> this through a  or a dedicated class that I inject. Not sure how
> the system is more secure by not allowing to call this method directly
> in this case.

You can do that, because you control the command line, an attacker may not be 
able to do the same thing.

Rémi


Re: Add reads to open module

2021-05-17 Thread Remi Forax
- Mail original -
> De: "Christian Beikov" 
> À: "jigsaw-dev" 
> Envoyé: Lundi 17 Mai 2021 17:43:15
> Objet: Fwd: Add reads to open module

> Hi,
> 
> I have a use case where I generate some code and that code refers to
> classes of some module B.
> 
> When the module A (an open module), into which I define that class, has
> no read-edges to the module B, this fails saying that A does not read B.
> 
> That's fine and totally understandable, so I tried to add the following
> code:
> 
> moduleA.addReads("moduleB.pkg", moduleB)
> 
> but this is not allowed as `addReads` only works if `this == callerModule`.
> 
> My workaround is to define a class into module A that can then call this
> method.
> 
> Is there a reason to this limitation? I mean the module is open anyway,
> so anyone can just define a class into it, so why not also allow calling
> addReads?

It's basic security [1], you can not see something you do not ask for.
By example, if your module as a security issue, you don't want an attacker to 
be able to see all other open modules from your module.

> 
> 
> Regards,
> 
> Christian

regards,
Rémi

[1] https://en.wikipedia.org/wiki/Defense_in_depth_(computing)


Re: Why service provider method is called "provider", but not "provide"?

2020-11-20 Thread Remi Forax
- Mail original -
> De: "Alex Sviridov" 
> À: "jigsaw-dev" 
> Envoyé: Vendredi 20 Novembre 2020 12:48:21
> Objet: Why service provider method is called "provider", but not "provide"?

> Hello all,

Hi Alex,

> 
> According to this tutorial
> https://www.logicbig.com/tutorials/core-java-tutorial/modules/service-provider-method.html
> we can declare in module-info provider class with "provider" method. For
> example,
> 
> ..
> public class TheServiceProvider {
>   public static AService provider() {
>   return new AServiceImpl();
>   }
> }
> 
> The only thing what I don’t understand is why this method is called 
> "provider".
> The method doesn’t return
> provider, method returns an instance of the service, so, as I understand, the
> method must be named
> "provide". And, as I understand, provider is usually the class, that has
> "provide" method.

The spec is in ServiceLoader
https://docs.oracle.com/en/java/javase/15/docs/api/java.base/java/util/ServiceLoader.html

yes, the name is not very logical.

> 
> Of course that’s not very important, but it is JDK, so it was developed by 
> very
> experienced java developer
> who should know java naming convention and I want to understand why this did 
> so.
> Could anyone explain?

I believe it's due to the fact that most services used by the JDK are 
themselves factories.
And naming is hard.

> 
> 
> --
> Best regards, Alex Orlov

regards,
Rémi


Re: Illegal reflection access denial in which future release

2020-06-08 Thread Remi Forax
- Mail original -
> De: "Claes Redestad" 
> À: "S A" , "jigsaw-dev" 
> 
> Cc: "Andrey Loskutov" 
> Envoyé: Lundi 8 Juin 2020 16:09:28
> Objet: Re: Illegal reflection access denial in which future release

> Hi Simeon,
> 
> it's not documented or discussed much anywhere because it's as-of-yet
> undecided.
> 
> *Expectation* is that there'll be a JEP proposed to make
> --illegal-access=deny the *default* in JDK 16. I expect the ability to
> explicitly permit illegal accesses with --illegal-access=permit will
> stick around for a good while longer, though.

As a member of the expert group on JPMS, i think we should move to deny by 
default for 16.
A lot of people are now migrating their applications to 11, i think it's 
cheaper to do the migration to a world with --illegal-access=deny than to do 
first a migration to 11 and then a migration to 17.

> 
> Best regards
> /Claes

regards,
Rémi

> 
> On 2020-06-08 15:40, S A wrote:
>> Hi all,
>> 
>> TL;DR is a future JDK release known, when illegal reflection access is no
>> longer possible? If not, are there estimations / discussions / FAQs /
>> documentation?
>> 
>> The product I work on (Eclipse+xtext IDE + backend) has nearly completed
>> the move to OpenJDK 11 (from OpenJDK 8) and we are working on supposedly
>> not pressing issues. One of those issues is illegal access via reflection.
>> 
>> We would obviously like to be prepared and so are estimating effort and
>> time constraints of fixing illegal reflection access. As this includes
>> multiple libraries, knowing *when* we need to finish our efforts is quite
>> important.
>> 
>> Sorry if this has been asked before (I imagine yes), but searching for the
>> topic yields lots of how to work around or fix the issue. Thank you in
>> advance.
>> 
>> Best regards,
>> Simeon


Re: Is it possible to add classes to JPMS module dynamically?

2020-06-04 Thread Remi Forax
- Mail original -
> De: "Alex Sviridov" 
> À: "jigsaw-dev" 
> Envoyé: Jeudi 4 Juin 2020 14:34:08
> Objet: Is it possible to add classes to JPMS module dynamically?

> Hi all,

Hi Alex,

> 
> Let's suppose we have a JPMS module  moduleA with the following  module-info :
> module moduleA {
>exports modulea.generated;
> }
> Now, using javassist/byte-buddy we generate new  modulea.generated.Foo class.
> Could anyone say if we can add this class to  moduleA dynamically (at runtime)
> so as JPMS work with it as it works with module existing classes ?

It will be added automatically, a module only contains packages (exported or 
not) so any classes in those packages (generated at runtime or not) are part of 
the module automatically.

> 
> 
> --
> Alex Orlov

cheers,
Rémi


Re: ModuleMainClass attribute via module-info.java

2020-05-31 Thread Remi Forax
- Mail original -
> De: "Florian Weimer" 
> À: "jigsaw-dev" 
> Envoyé: Dimanche 31 Mai 2020 19:31:48
> Objet: ModuleMainClass attribute via module-info.java

> Is it possible to instruct javac to generate a ModuleMainClass
> attribute via a module declaration?  I can't find the appropriate
> syntax in the JLS.  If there is no way to get javac to generate the
> attribute, what is the reason for that?

Hi Florian,
the design of JPMS separates what is useful for Java described by the 
module-info.java (required modules, exported or open packages, etc) and what is 
metadata for distribution (module version, module main, modules hashes, etc) 
stored as supplementary attributes (or optional fields of the Module attribute) 
in the module-info.class.

Changing the main class or the version of a module do not require a 
recompilation, so those information are not handled by javac but injected into 
the module-info.class by jar, because those information are useful only when 
you want to create a jar in order to distribute it.

We came to that design to be able to support already existing ways of packaging 
the metadata associated with a jar file so a jar can be at the same time a 
module for JPMS and also a module for Jboss module or an OSGI bundle, etc. As 
an example of metadata fusion, you can use a tool like bnd [1], a tool 
developed originally to package OSGI bundles, if you want to use a Java 
annotation to specify the main class. 

regards,
Rémi

[1] https://bnd.bndtools.org/chapters/330-jpms.html


Re: jlink 14.0.1 with 100 modules fail

2020-05-30 Thread Remi Forax
- Mail original -
> De: "Christian Stein" 
> À: "jigsaw-dev" 
> Envoyé: Samedi 30 Mai 2020 18:40:53
> Objet: jlink 14.0.1 with 100 modules fail

> Hi,
> 
> the modular [bach-demo-99] I generated to check my build tool
> against, failed to create a custom runtime image using jlink
> via itsToolProvider-based service entry-point.
> 
> The project consists of 100 module descriptors. No other Java
> source files (for classes and interfaces) are part of that demo.
> Each descriptor ball${i}n declares a "requires ball${j}n;" with j
> set to a number from 0 to i-1 ... in words, all lower modules.
> 
> javac compiles this project with problems.
> 
> Arguments passed to jlink (as an array of string) reads:
> 
> jlink
>  --add-modules ball27n,ball43n,ball88n,[...],ball77n,ball93n
>  --module-path .bach\workspace\modules
>  --output .bach\workspace\image
>  --compress 2
>  --no-header-files
>  --no-man-pages
> 
> The error message reads:
> 
> jdk.internal.org.objectweb.asm.MethodTooLargeException: Method too large:
> jdk/internal/module/SystemModules$all.moduleDescriptors
> ()[Ljava/lang/module/ModuleDescriptor;
> 
> Is there a limitation of supported modules that can be linked?

It's a bug in the implementation,
jlink pre-resolve the module graph and encodes it in a generated classfile.
The fact that that classfile can have a method too big was overlooked. 

> Is the module graph of this conceived project too far out of scope?
> 
> Shall I open an issue at JBS?


yes !

> 
> Cheers,
> Christian

regards,
Rémi

> 
> [ bach-demo-99]:
> https://github.com/sormuras/bach/blob/master/src/bach/bach-demo-99.jsh


Re: replacements for Unsafe

2020-04-29 Thread Remi Forax
Hi Jochen,
Per the Java spec, calling an empty static method of the inner class works.

Rémi

- Mail original -
> De: "Jochen Theodorou" 
> À: "jigsaw-dev" 
> Envoyé: Mercredi 29 Avril 2020 13:23:47
> Objet: replacements for Unsafe

> Hi,
> 
> when jigsaw started there was a lot of talk about providing alternatives
> to what Unsafe offers. Today I am facing a problem I cannot solve
> anymore it seems with this.
> 
> My problem is that I need to ensure the static block of an inner class
> has been executed before I do something, because the static block
> contains setup data the class will need and influence what I do with the
> class. Of course this might be not the best choice, I am just trying to
> keep certain functionality.
> 
> In the past I would have used Unsafe#ensureClassInitialized, but this
> method has not found a replacement yet if I read JDK-8235521 :
> Replacement API for Unsafe::ensureClassInitialized correctly.
> 
> Also because of Unsafe#getUnsafe, checking for the platform loader for
> the caller (which is not the case for me) I cannot get the unsafe object
> itself "properly".
> 
> Now I am wondering what alternatives exist. Are there any?
> 
> bye Jochen


Re: excluding transitive module

2020-04-15 Thread Remi Forax
- Mail original -
> De: "Jochen Theodorou" 
> À: "jigsaw-dev" 
> Envoyé: Mercredi 15 Avril 2020 08:55:00
> Objet: Re: excluding transitive module

> On 14.04.20 20:20, Remi Forax wrote:
>> Hi Jochen,
>> JPMS has no notion of of API and implementation of the same jar. It's a 
>> concept
>> of your build tool and not something JPMS knows.
>>
>> The notion of compilation dependencies and runtime dependencies is not a 
>> concept
>> of JPMS but a concept of your build tools.
>>
>> In term of JPMS, if you want to be able to use a jar at compile time and 
>> another
>> one at runtime,
>> both should have the same module name, in fact they should have exactly the 
>> same
>> module descriptor (the same module-info).
>>
>> So in term of dependency,
>> - for,Maven/Gradle, the compile dependency will use SharedAPI and the runtime
>> dependency SharedApiImpl
>> - for JPMS, both have the same module-info and if you want to depend on that
>> module, just require it.
>>
>>  From the Maven Central POV, obviously, you have two jars so they can not 
>> have
>>  the same arfifact ID (coordinate), but they contains the same
>>  module-info.class.
> 
> If the case is api and implementation and the same module name, then
> using the build tool to exclude the api dependency works.
> 
> If the module name is not the same I have not found a solution. It is a
> problem here because of shared packages.

as Alex said, this should be fixed by the maintainers of the jars, not you.

> 
> I found this being a problem btw with a lot of jee libraries.
> 
> Of course I can make my project require whatever, but if it is libraries
> I do not have under control things get a bit difficult without changing
> them. But changing them is not an action really well supported by any
> build tool I know.

For the Jakarta EE libraries, the main issue is that there are not many jars 
that has a module-info, so the default strategy to name the automatic module 
with the jar name fail here because an api an its implementations should have 
the same name.
Injecting the right module-info until your pull request is accepted is an 
escape hatch.

> 
> bye Jochen

Rémi


Re: excluding transitive module

2020-04-14 Thread Remi Forax
Hi Jochen,
JPMS has no notion of of API and implementation of the same jar. It's a concept 
of your build tool and not something JPMS knows.

The notion of compilation dependencies and runtime dependencies is not a 
concept of JPMS but a concept of your build tools.

In term of JPMS, if you want to be able to use a jar at compile time and 
another one at runtime,
both should have the same module name, in fact they should have exactly the 
same module descriptor (the same module-info). 

So in term of dependency,
- for,Maven/Gradle, the compile dependency will use SharedAPI and the runtime 
dependency SharedApiImpl
- for JPMS, both have the same module-info and if you want to depend on that 
module, just require it.

>From the Maven Central POV, obviously, you have two jars so they can not have 
>the same arfifact ID (coordinate), but they contains the same 
>module-info.class.

regards,
Rémi

- Mail original -
> De: "Jochen Theodorou" 
> À: "jigsaw-dev" 
> Envoyé: Mardi 14 Avril 2020 10:24:34
> Objet: excluding transitive module

> Hi all,
> 
> I am wondering if there is a solution purely in the module-info for this:
> 
> * Project requires Library1 and Library2
> * SomeLibrary requires SharedApi
> * OtherLibrary requires SharedApiImpl
> 
> The problem is, that it will not compile because SharedApi and
> SharedApiImpl implement the same packages but not the same logic. One is
> just an API, while the other is an implementation of the API.
> 
> Scenario 1:
> 
> What would seem to be logic to me is to exclude SharedApi, but does
> SomeLibrary then need its requires to be changed to SharedApiImpl? How
> would I do that if I cannot change SomeLibrary?
> 
> Scenario 2:
> 
> Let us assume I could change the module-info of SomeLibrary. How would I
> express that Shared API is compilation only? Is that what "requires
> SomeLibrary for compilation" would be for?
> 
> bye Jochen


Re: RFE simplify usage of patched module [by Robert Scholte, from jdk-dev]

2020-02-25 Thread Remi Forax
- Mail original -
> De: "Alex Buckley" 
> À: "jigsaw-dev" 
> Envoyé: Mardi 18 Février 2020 18:42:10
> Objet: Re: RFE simplify usage of patched module [by Robert Scholte, from 
> jdk-dev]

> On 2/14/2020 9:34 PM, Christian Stein wrote:
>> Assuming `--patch-module-descriptor` was already available, the
>> org.astro test story would read:
>>
>> - org.astro/main/module-info.java:
>>
>> module org.astro {
>>exports org.astro;
>>requires java.logging;
>> }
>>
>> - org.astro/test/module-info.java:
>>
>> open /*test*/ module org.astro /*extends main module org.astro*/ {
>>requires java.sql; // system-provided module needed by test code
>>requires org.junit.jupiter; // framework reflecting entry-points
>>requires org.assertj.core; // ...and more test-related modules
>> }
>>
>> A build tool now easily may detect that `--patch-module-descriptor`
>> is required at test compile and run time. It could infer it from the
>> same module name or simply by seeing a main and a test
>> `module-info.java` compilation unit within the sources of the
>> org.astro Maven-/Gradle-/pro-/Bach-/any-build-tool-"module".
>>
>> Build tools could pass the following argument to javac/java
>> while being in their test-scope tasks. Of course, implying that
>> org.astro/test/module-info.java is the current "primary" module
>> descriptor:
>>
>> --patch-module-descriptor org.astro=org.astro/main/module-info.java
>>
>> That's it.
>>
>> The primary test-relevant modular directives are enriched by
>> those of the main module as declared in org.astro/main/module-info.java
>> resulting effectively in a synthetic in-module test module descriptor that
>> resembles the inter-module test descriptor presented above:
>>
>> - SYNTHETIC test module for org.astro
>>
>> open /*test*/ module org.astro /*extends main module org.astro*/ {
>>requires java.sql; // system-provided module needed by test code
>>requires org.junit.jupiter; // framework reflecting entry-points
>>requires org.assertj.core; // ...and more test-related modules
>>//
>>exports org.astro; // patched from org.astro/main/module-info.java
>>requires java.logging; // patched from org.astro/main/module-info.java
>> }
>
> Good description of how the main module
> (org.astro/main/module-info.java) and the test module
> (org.astro/test/module-info.java) interact to form a test-scoped module.
> If a build tool has detected the presence of the test module, then the
> build tool can use javax.lang.model.element.ModuleElement to inspect the
> test module's declaration and infer what it requires, whether it's open,
> etc. Then, the build tool can enrich the main module by passing
> --add-modules, --add-reads, and --add-opens to javac/java while in a
> test scope.

Hi Alex,
no you can't because you can not reconstruct a module-info from the flags 
--add-modules, --add-reads, etc.

Those flags has been created to patch (mostly the JDK) module metadata so an 
application using the classpath can open/patch the JDK modules metadata enough 
to be able to run.
The semantics of a module-info is richer than the semantics of the flags, by 
example, there is no way to define a transitive dependency or to define a 
service.
I believe that roughly half of the semantics of a module-info has no equivalent 
in term of flags (and for good reason, those flags have not been created to 
describe a module-info).

> No need for --patch-module-descriptor in the JDK.

We need a way to express that the module graph when testing is different from 
the one when running in production.
--path-module-descriptor is a way to get that.

>
> Alex

Rémi


Re: Lookup objects and setAccessible

2020-01-08 Thread Remi Forax
see 
https://docs.oracle.com/en/java/javase/13/docs/api/java.base/java/lang/invoke/MethodHandles.html#privateLookupIn(java.lang.Class,java.lang.invoke.MethodHandles.Lookup)

Rémi

- Mail original -
> De: "Kasper Nielsen" 
> À: "jigsaw-dev" 
> Envoyé: Mercredi 8 Janvier 2020 12:14:30
> Objet: Lookup objects and setAccessible

> Hi,
> 
> I'm trying to bridge some old code using java.lang.reflect and some new code
> that uses Lookup objects. And I'm wondering if there is some way to make
> a member accessible using a Lookup object? Or if open packages/modules via
> module-info.java + member.setAccessible(true) is the only way?
> 
> Thanks
>   Kasper


Re: Could we get module-private visibility?

2019-09-25 Thread Remi Forax
- Mail original -
> De: "Mike Hearn" 
> À: "Michael Rasmussen" 
> Cc: "jigsaw-dev" 
> Envoyé: Mercredi 25 Septembre 2019 11:30:00
> Objet: Re: Could we get module-private visibility?

> Yes, I know. I'm talking about module-private inside exported packages.
> Otherwise it can be quite difficult to allow exported and non-exported
> packages to work closely together without weakening visibility, and at any
> rate, whilst the compatibility considerations are obvious, having a
> visibility level of "public" that's actually private to a scope is not a
> design anyone would pick if they had a blank slate I imagine!

at the same time, a public method is not really public if the class is not 
public too,
so it's like russian dolls all the way down.

Rémi



Re: Qualified exports to an unknown module

2018-12-08 Thread Remi Forax



- Mail original -
> De: "Alan Bateman" 
> À: "jigsaw-dev" 
> Envoyé: Samedi 8 Décembre 2018 09:44:05
> Objet: Re: Qualified exports to an unknown module

> On 07/12/2018 18:59, Alex Buckley wrote:
>>
>> The decision was: Qualified exports/opens to a non-observable module
>> is NOT an error (see
>> https://docs.oracle.com/javase/specs/jls/se11/html/jls-7.html#jls-7.7.2-310).
>> javac will give an on-by-default lint warning instead.
>>
>> The StackOverflow question says "Because I configured the compiler to
>> treat warnings as errors (-Werror), the build fails." so the
>> questioner should look at disabling the lint warning (see
>> https://docs.oracle.com/en/java/javase/11/tools/javac.html#GUID-AEEC9F07-CB49-4E96-8BC7-BCC2C7F725C9).
> The scenario in the post seems to be a Maven multi-module project so an
> alternative to -Xlint:-module is to compile with --module-source-path so
> that module B is observable when compiling A. This should work:
>     javac --module-source-path project/"*"/src/main/java -d  -m A
> 
> although I don't know if the Maven compiler plugin can be configured to
> use this option/syntax.

Maven still can not do that, which is a bummer because compiling in 
multi-module mode is also faster when you starts to have quite a lot of classes.

> 
> -Alan

Rémi


Re: Separate logging for JPMS module/layer

2018-10-04 Thread forax
I was thinking about capturing the call stack when you create the logger (to 
get the module), not when you call the logger.

cheers,
Rémi

- Mail original -
> De: "Ralph Goers" 
> À: "Alex Sviridov" 
> Cc: "Remi Forax" , "jigsaw-dev" 
> 
> Envoyé: Mercredi 3 Octobre 2018 05:08:27
> Objet: Re: Separate logging for JPMS module/layer

> Log4j handles this by capturing the fully qualified class name of the logging
> adapter. Obviously, this doesn’t work if the adapter doesn’t pass Log4j the
> FQCN, but it does work for the adapters we support.  That said, it is very 
> slow
> to capture this and is probably the biggest pain point. Log4j recommends not
> capturing this information in production environments because it is so slow.
> Unfortunately, it seems to have gotten even slower in Java 9+. In an ideal
> world we would be able to capture the caller information at compile time but
> Java provides no good way to do this. Wouldn’t it be great if I could just 
> code
> something like logger.error(_CallerInfo_, “hello”) and the compiler would
> provide the caller info data structure that was generated by the compiler?
> 
> FWIW, I do plan to add the module information to the caller information 
> provided
> with Log4j but just haven’t gotten to it. You are more than welcome to provide
> a patch.
> 
> Ralph
> 
>> On Oct 2, 2018, at 3:20 PM, Alex Sviridov  wrote:
>> 
>> Thank you for you suggestion. But can this be used when some library
>> uses one logging system and for another uses some bridge. Because of this
>> bridging
>> LoggerFactory.getLogger is called somewhere in bridge, as I understand,
>> 
>> 
>>> Среда,  3 октября 2018, 1:12 +03:00 от Remi Forax :
>>> 
>>> You can use the StackWalker
>>> https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/StackWalker.html
>>> 
>>> regards,
>>> Rémi
>>> 
>>> - Mail original -
>>>> De: "Alex Sviridov" < ooo_satu...@mail.ru >
>>>> À: "jigsaw-dev" < jigsaw-dev@openjdk.java.net >
>>>> Envoyé: Mardi 2 Octobre 2018 23:54:48
>>>> Objet: Separate logging for JPMS module/layer
>>> 
>>>> Hi all,
>>>> 
>>>> Could anyone say how the following problem can be solved. I want to create
>>>> separate
>>>> log file for every JPMS module/layer. The problem is that many
>>>> libraries/programs
>>>> use LoggerFactory.getLogger(String className) so in getLogger I have only
>>>> the name of the class as String, so I can't get module and layer.
>>>> 
>>>> If I had not String className, but Class klass then the problem would be 
>>>> easily
>>>> solved.
>>>> As I understand I can't load class by name because it would require all 
>>>> modules
>>>> export
>>>> their packages to logging framework that has no sense.
>>>> 
>>>> Are there any solutions for such problem?
>>>> 
>>>> 
>>>> --
>>>> Alex Sviridov
>> 
>> 
>> --
> > Alex Sviridov


Re: Re[2]: Separate logging for JPMS module/layer

2018-10-02 Thread forax
> De: "Alex Sviridov" 
> À: "Remi Forax" , "jigsaw-dev" 
> 
> Envoyé: Mercredi 3 Octobre 2018 00:20:08
> Objet: Re[2]: Separate logging for JPMS module/layer

> Thank you for you suggestion. But can this be used when some library
> uses one logging system and for another uses some bridge. Because of this
> bridging
> LoggerFactory.getLogger is called somewhere in bridge, as I understand,
You can still skip over the method of the bridge, but usually it only works if 
you know all the bridges that can happen because this code tend to be depend on 
the implementation of each bridge, 
practically, there is less than half a dozen of logging libraries used in Java, 
so it's doable. 

The other solution is to ask every maintainers of the logging libraries to 
implement the System.Logger API introduced in 9. 

Rémi 

>> Среда, 3 октября 2018, 1:12 +03:00 от Remi Forax :

>> You can use the StackWalker
>> [
>> https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/StackWalker.html
>> |
>> https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/StackWalker.html
>> ]

>> regards,
>> Rémi

>> - Mail original -
>> > De: "Alex Sviridov" < [ mailto:ooo_satu...@mail.ru | ooo_satu...@mail.ru ] 
>> > >
>>> À: "jigsaw-dev" < [ mailto:jigsaw-dev@openjdk.java.net |
>> > jigsaw-dev@openjdk.java.net ] >
>> > Envoyé: Mardi 2 Octobre 2018 23:54:48
>> > Objet: Separate logging for JPMS module/layer

>> > Hi all,

>> > Could anyone say how the following problem can be solved. I want to create
>> > separate
>> > log file for every JPMS module/layer. The problem is that many
>> > libraries/programs
>> > use LoggerFactory.getLogger(String className) so in getLogger I have only
>> > the name of the class as String, so I can't get module and layer.

>> > If I had not String className, but Class klass then the problem would be 
>> > easily
>> > solved.
>> > As I understand I can't load class by name because it would require all 
>> > modules
>> > export
>> > their packages to logging framework that has no sense.

>> > Are there any solutions for such problem?


>> > --
>> > Alex Sviridov

> --
> Alex Sviridov


Re: Separate logging for JPMS module/layer

2018-10-02 Thread Remi Forax
You can use the StackWalker
https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/StackWalker.html

regards,
Rémi

- Mail original -
> De: "Alex Sviridov" 
> À: "jigsaw-dev" 
> Envoyé: Mardi 2 Octobre 2018 23:54:48
> Objet: Separate logging for JPMS module/layer

> Hi all,
> 
> Could anyone say how the following problem can be solved. I want to create
> separate
> log file for every JPMS module/layer. The problem is that many
> libraries/programs
> use LoggerFactory.getLogger(String className) so in getLogger I have only
> the name of the class as String, so I can't get module and layer.
> 
> If I had not String className, but Class klass then the problem would be 
> easily
> solved.
> As I understand I can't load class by name because it would require all 
> modules
> export
> their packages to logging framework that has no sense.
> 
> Are there any solutions for such problem?
> 
> 
> --
> Alex Sviridov


Re: Add posibility to add custom ModuleReaderFactory to ModuleFinder

2018-09-28 Thread Remi Forax



- Mail original -
> De: "Alan Bateman" 
> À: "Alex Sviridov" , "jigsaw-dev" 
> 
> Envoyé: Vendredi 28 Septembre 2018 15:51:56
> Objet: Re: Add posibility to add custom ModuleReaderFactory to ModuleFinder

> On 28/09/2018 13:10, Alex Sviridov wrote:
>> Hi Alan
>>
>> Thank you for your answer. But my main problem is not jars inside .war
>> - this is a so far from my current problem. Now I need to 1) add .war
>> file to layer 2). to map file location, for example instead of
>> "module-info.java" I must find "WEB-INF/classes/module-info.java" etc.
>> That is all I need. How can I do it without implementing ModuleFinder?
> You'll need a ModuleFinder because the packaging formats that
> ModuleFinder.of(Path) is required to support doesn't know anything about
> WAR files. It's not super difficult to develop your own. I attach a
> simple implementation that may get you started. It's really basic but
> would need a few iterations to be robust. Invoke
> WarModuleFinder.of(Path) with the file path to the WAR file and it will
> create a ModuleFinder that can find the application module in the WAR
> file. A more complete implementation would be a lot more robust and
> polished that this sample, it would also find the modules WEB-INF/lib.
> 
> Once you have a ModuleFinder then you specify it to Conguration::resolve
> method when resolving the application as the root module. You'll
> probably start with something like:
> 
>     Path war = Path.of("app.war");
>     ModuleFinder finder = WarModuleFinder.of(war);
> 
>     String appModuleName = finder.findAll().stream()
>     .findFirst()
>     .map(ModuleReference::descriptor)
>     .map(ModuleDescriptor::name)
>     .orElseThrow();
> 
>     ModuleLayer boot = ModuleLayer.boot();
>     Configuration cf = boot.configuration().resolve(finder,
> ModuleFinder.of(), Set.of(appModuleName));
>     ModuleLayer layer = boot.defineModulesWithOneLoader(cf,
> ClassLoader.getSystemClassLoader());
> 
> and now you have a module layer with the application module loaded from
> the WEB-INF/classes part of the WAR file.
> 
> -Alan
> 
> 
>     static class WarModuleFinder implements ModuleFinder {
>     private final FileSystem warfs;
>     private final Path classes;
>     private final ModuleReference mref;
> 
>     private WarModuleFinder(Path warfile) throws IOException {
>     ClassLoader scl = ClassLoader.getSystemClassLoader();
>     FileSystem fs = FileSystems.newFileSystem(warfile, scl);
>     Path classes = fs.getPath("/WEB-INF/classes");
> 
>     ModuleDescriptor descriptor;
>     try (InputStream in =
> Files.newInputStream(classes.resolve("module-info.class"))) {
>     descriptor = ModuleDescriptor.read(in, () ->
> packages(classes));
>     }
> 
>     this.warfs = fs;
>     this.classes = classes;
>     this.mref = new ModuleReference(descriptor, classes.toUri()) {
>     @Override
>     public ModuleReader open() {
>     return new WarModuleReader();
>     }
>     public String toString() {
>     StringBuilder sb = new StringBuilder();
>     sb.append("[module ");
>     sb.append(descriptor().name());
>     sb.append(", location=");
>     sb.append(location());
>     sb.append("]");
>     return sb.toString();
>     }
>     };
>     }
> 
>     static WarModuleFinder of(Path war) throws IOException {
>     return new WarModuleFinder(war);
>     }
> 
>     @Override
>     public Optional find(String name) {
>     if (name.equals(mref.descriptor().name())) {
>     return Optional.of(mref);
>     } else {
>     return Optional.empty();
>     }
>     }
> 
>     @Override
>     public Set findAll() {
>     return Set.of(mref);
>     }
> 
>     private Set packages(Path classes) {
>     try {
>     return Files.find(classes, Integer.MAX_VALUE,
>   (path, attrs) -> !attrs.isDirectory())
>     .map(entry -> classes.relativize(entry).toString())
>     .map(this::toPackageName)
>     .flatMap(Optional::stream)
>     .collect(Collectors.toSet());
>     } catch (IOException ioe) {
>     throw new UncheckedIOException(ioe);
>     }
>     }
> 
>     private Optional toPackageName(String name) {
>     int index = name.lastIndexOf("/");
>     if (index > 0) {
>     return Optional.of(name.substring(0,
> index).replace('/', '.'));
>     } else {
>     return Optional.empty();
>     }
>     }
> 
>   

Re: method handles and open packages

2018-07-21 Thread forax
- Mail original -
> De: "Michał Zegan" 
> À: "Remi Forax" , "jigsaw-dev" 
> 
> Envoyé: Samedi 21 Juillet 2018 21:44:23
> Objet: Re: method handles and open packages

> I would also believe that the unreflect methods would work for this case
> too if the reflected version had access checks already suppressed.

yes, right !
if the signature of the method is fully known, you can use privateLookupIn,
if you have only partial information and need other info (like an annotation, 
using a naming convention, etc) to find the method, then unreflect* is the way 
to go.
 
Rémi

> 
> W dniu 21.07.2018 o 19:27, Remi Forax pisze:
>> Hi,
>> 
>> On July 21, 2018 3:43:51 PM UTC, "Michał Zegan" 
>> wrote:
>>> Hello,
>>> If I use reflection, I can access every member of every package in a
>>> module if a module opens such package, and that includes private
>>> members.
>>> What, however, about using method handles for that purpose?
>> 
>> https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/MethodHandles.html#privateLookupIn-java.lang.Class-java.lang.invoke.MethodHandles.Lookup-
>> 
>>> Also, if you have a class in module a that wants to deserialize data to
>>> objects of classes in module b, then should module b open a package
>>> with
>>> those classes to a, or maybe, give the deserializer it's lookup object,
>>> in order for it to be able to access members of non exported non opened
>>> packages? It is for the case where the deserializer is called by the
>>> module b. Also I assume that I do not need access to non public members
>>> of classes, like I deserialize using public setters always.
>> 
>> Rémi
>> 
>> 


Re: method handles and open packages

2018-07-21 Thread Remi Forax
Hi,

On July 21, 2018 3:43:51 PM UTC, "Michał Zegan"  
wrote:
>Hello,
>If I use reflection, I can access every member of every package in a
>module if a module opens such package, and that includes private
>members.
>What, however, about using method handles for that purpose?

https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/MethodHandles.html#privateLookupIn-java.lang.Class-java.lang.invoke.MethodHandles.Lookup-

>Also, if you have a class in module a that wants to deserialize data to
>objects of classes in module b, then should module b open a package
>with
>those classes to a, or maybe, give the deserializer it's lookup object,
>in order for it to be able to access members of non exported non opened
>packages? It is for the case where the deserializer is called by the
>module b. Also I assume that I do not need access to non public members
>of classes, like I deserialize using public setters always.

Rémi



-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.


Re: ClassLoader.getResources vs Module.getResourceAsStream

2018-07-16 Thread Remi Forax



- Mail original -
> De: "Sander Mak" 
> À: "jigsaw-dev" 
> Envoyé: Lundi 16 Juillet 2018 08:52:41
> Objet: Re: ClassLoader.getResources vs Module.getResourceAsStream

> In that case you'd expose the factories through the services mechanism
> (`provides com.acme.api.MyWidgetFactory with
> com.acme.factories.XmlBasedWidgetFactory` in the module descriptor). Or, if 
> you
> must expose the XML itself to the outside world rather than the factories, you
> can create a service that offers the XML as String or InputStream. Each module
> then has an implementation reading its own encapsulated XML. The former
> approach is preferable IMO.
> 
> Note that you can also use the services mechanism without module descriptors 
> by
> placing text files in META-INF/services
> (https://docs.oracle.com/javase/tutorial/ext/basics/spi.html). However, if 
> your
> code isn't a reusable library itself, you can also choose to modularize your
> own code and depend on non-modularized dependencies through their automatic
> module name
> (http://branchandbound.net/blog/java/2017/12/automatic-module-name/).
> 

You have to do both, each service declared in a module-info MUST be declared in 
META-INF/services too because you can use a modular jar in the classpath as a 
plain old jar. 

Alan, we should patch jar to warn when there is a module-info.class that 
declare services with no corresponding META-INF/services.

> Sander
> 

Rémi

> On 14 Jul 2018, at 17:58, Michał Zegan
> mailto:webczat_...@poczta.onet.pl>> wrote:
> 
> It is a completely new code. It is not modularized for now because some
> dependencies are not modularized, but I want it to be compatible to ease
> later modularization.
> My actual goal is to load xml files defining and configuring some
> factories, from all modules that contain them. Not sure how would you do
> that with services?
> 
> W dniu 14.07.2018 o 17:53, Alan Bateman pisze:
> On 14/07/2018 16:38, Michał Zegan wrote:
> What is then a recommendation for searching for all resources with name
> x that i can access? Something like load all configuration including
> some kind of extensions. I cannot list resources easily.
> 
> Services is the cleaner way to do this kind of thing, esp. if you have
> flexibility to change the code and move away from legacy ClassLoader
> getResources.
> 
> -Alan


The baby and the bathwater (the return)

2018-06-03 Thread Remi Forax
Hi all,
There were discussions on that list [1] about the fact that beginning with Java 
9, there were 2 ways to deploy modules, classpath vs module-path.

I've discovered last Friday there that are not 2 configurations but 3 
configurations.
You can also use jlink [2] and in that case, the modules are not loaded though 
the module-path but are considered as system modules, so a library should also 
be tested with that configuration.

In my case, Spring Boot annotations scanning works with the classpath, works 
with the module-path but fails if deployed as system modules [3].

regards,
Rémi


[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2018-March/013689.html
[2] http://openjdk.java.net/jeps/282
[3] https://github.com/forax/pro-demo/tree/master/spring-demo


Re: Compiling module descriptors is not reproducible?

2018-05-19 Thread Remi Forax
- Mail original -
> De: "Alan Bateman" 
> À: "org openjdk" , "jigsaw-dev" 
> 
> Envoyé: Samedi 19 Mai 2018 13:26:55
> Objet: Re: Compiling module descriptors is not reproducible?

> On 19/05/2018 10:18, Mark Raynsford wrote:
>> Hello!
>>
>> I've been interested for a while in making all of my software builds
>> reproducible [0]. I don't think there's ever been any kind of
>> written guarantee that the output of javac will be completely
>> deterministic, but to date it seems like it actually has been. However,
>> I was a bit disappointed today to find that the output of one of the
>> builds at AdoptOpenJDK [1] seems to be inserting compiler version
>> information into compiled module descriptors. It doesn't seem to be
>> doing this for any other class files.
> The requires table in the Module attribute can be used by compilers to
> record version information about dependences. javac does record the
> version information, I don't know if there is a way to disable that. 

It seems there is no way:
http://hg.openjdk.java.net/jdk/jdk/file/5ec7380f671d/src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassWriter.java#l986

> In any case, I assume your question will be partly answered by looking at
> the output of `java --list-modules` on the different builds. If you
> download OpenJDK builds from jdk.java.net/10 then you should see that
> the standard and JDK-specific modules all report their version string as
> "10" or "10.0.1" as they have been built with configure options that
> make it so. At a guess, your mail may have included the output from an
> exploded (as opposed to images) build.
> 
> -Alan

Rémi


Re: Compiling module descriptors is not reproducible?

2018-05-19 Thread Remi Forax
- Mail original -
> De: "org openjdk" 
> À: "jigsaw-dev" 
> Envoyé: Samedi 19 Mai 2018 11:18:09
> Objet: Compiling module descriptors is not reproducible?

> Hello!

Hi Mark,

> 
> I've been interested for a while in making all of my software builds
> reproducible [0]. I don't think there's ever been any kind of
> written guarantee that the output of javac will be completely
> deterministic, but to date it seems like it actually has been. However,
> I was a bit disappointed today to find that the output of one of the
> builds at AdoptOpenJDK [1] seems to be inserting compiler version
> information into compiled module descriptors. It doesn't seem to be
> doing this for any other class files.
> 
> Here's the output of "openjdk version 10.0.1 2018-04-17" on Linux
> (not an AdoptOpenJDK build) when compiling a module descriptor:
> 
> ---
> module com.io7m.jtensors.core
>  minor version: 0
>  major version: 53
>  flags: (0x8000) ACC_MODULE
>  this_class: #1  // "module-info"
>  super_class: #0
>  interfaces: 0, fields: 0, methods: 0, attributes: 2
> Constant pool:
>   #1 = Class  #17// "module-info"
>   #2 = Utf8   SourceFile
>   #3 = Utf8   module-info.java
>   #4 = Utf8   Module
>   #5 = Module #18// "com.io7m.jtensors.core"
>   #6 = Module #19// "java.base"
>   #7 = Module #20// "org.immutables.value"
>   #8 = Module #21// "com.io7m.junreachable.core"
>   #9 = Package#22// 
> com/io7m/jtensors/core/determinants
>  #10 = Package#23// com/io7m/jtensors/core/dotproducts
>  #11 = Package#24//
>  com/io7m/jtensors/core/parameterized/matrices
>  #12 = Package#25//
>  com/io7m/jtensors/core/parameterized/vectors
>  #13 = Package#26// com/io7m/jtensors/core/quaternions
>  #14 = Package#27//
>  com/io7m/jtensors/core/unparameterized/matrices
>  #15 = Package#28//
>  com/io7m/jtensors/core/unparameterized/vectors
>  #16 = Package#29// com/io7m/jtensors/core
>  #17 = Utf8   module-info
>  #18 = Utf8   com.io7m.jtensors.core
>  #19 = Utf8   java.base
>  #20 = Utf8   org.immutables.value
>  #21 = Utf8   com.io7m.junreachable.core
>  #22 = Utf8   com/io7m/jtensors/core/determinants
>  #23 = Utf8   com/io7m/jtensors/core/dotproducts
>  #24 = Utf8   com/io7m/jtensors/core/parameterized/matrices
>  #25 = Utf8   com/io7m/jtensors/core/parameterized/vectors
>  #26 = Utf8   com/io7m/jtensors/core/quaternions
>  #27 = Utf8   com/io7m/jtensors/core/unparameterized/matrices
>  #28 = Utf8   com/io7m/jtensors/core/unparameterized/vectors
>  #29 = Utf8   com/io7m/jtensors/core
> {
> }
> SourceFile: "module-info.java"
> Module:
>  #5,0// "com.io7m.jtensors.core"
>  #0
>  3   // requires
>#6,8000 // "java.base" ACC_MANDATED
>#0
>#7,40   // "org.immutables.value"
>ACC_STATIC_PHASE
>#0
>#8,0// "com.io7m.junreachable.core"
>#0
>  8   // exports
>#9,0// 
> com/io7m/jtensors/core/determinants
>#10,0   // 
> com/io7m/jtensors/core/dotproducts
>#11,0   //
>com/io7m/jtensors/core/parameterized/matrices
>#12,0   //
>com/io7m/jtensors/core/parameterized/vectors
>#13,0   // 
> com/io7m/jtensors/core/quaternions
>#14,0   //
>com/io7m/jtensors/core/unparameterized/matrices
>#15,0   //
>com/io7m/jtensors/core/unparameterized/vectors
>#16,0   // com/io7m/jtensors/core
>  0   // opens
>  0   // uses
>  0   // provides
> ---
> 
> Here's the same module descriptor compiled on "openjdk version
> 10-internal" on MacOS (this is actually OpenJDK 10+23):
> 
> ---
> module com.io7m.jtensors.core
>  minor version: 0
>  major version: 53
>  flags: (0x8000) ACC_MODULE
>  this_class: #1  // "module-info"
>  super_class: #0
>  interfaces: 0, fields: 0, methods: 0, attributes: 2
> Constant pool:
>   #1 = Class  #18// "module-info"
>   #2 = Utf8   SourceFile
>   #3 = Utf8   module-info.java
>   #4 = Utf8   Module
>   

Re: The baby and the bathwater

2018-03-28 Thread Remi Forax
yes, 
!Whatever.class.getModule().isNamed()

Rémi

- Mail original -
> De: "Peter Levart" 
> À: "mark reinhold" , "jigsaw-dev" 
> 
> Envoyé: Mercredi 28 Mars 2018 09:28:23
> Objet: Re: The baby and the bathwater

> On 03/26/18 20:08, mark.reinh...@oracle.com wrote:
>> Stephen closes with a specific suggestion:
>>
>>"There needs to be a way for a library author to insist that the
>> modular jar file they are producing can only be run on the module-path
>> (with any attempt to use it on the class-path preventing application
>> startup).  This would eliminate the need for testing both class-path
>> and module-path."
> 
> That's easy to enforce in runtime. Just take a "victim" class from your
> library that is most often needed when your library is being used (or
> take a couple of them) and add a class initialization block like the
> following to them:
> 
> public class Whatever {
> 
>     static {
>         if (Whatever.class.getModule().getName() == null) {
>             throw new Error("Can only use this library as a module");
>         }
>     }
> 
> 
> Regards, Peter


Re: The baby and the bathwater

2018-03-27 Thread Remi Forax


- Mail original -
> De: "Alan Bateman" 
> À: "Cédric Champeau" 
> Cc: "jigsaw-dev" 
> Envoyé: Mardi 27 Mars 2018 10:29:36
> Objet: Re: The baby and the bathwater

> On 27/03/2018 08:15, Cédric Champeau wrote:
>> Dual testing is a minimum. In practice, it depends on the kind of tests.
>> Typically, before JDK 9 for unit tests you never needed a jar to execute
>> unit tests. Maven happens to built it, but in practice a class directory +
>> resources is enough
> This hasn't changed. You can put directories containing the test classes
> + resources on the class path as before. When testing modules you can
> patch a module to add the test classes (and resources) that are compiled
> into a directory, no need for either the module or the tests to be
> packaged as JAR files.

with the limitation that you can not patch a module-info so if you have 
testing-only dependencies like JUnit and you want to run them in module-mode, 
you have to generate a jar.

> 
> Maybe your comment is about testing libraries that are Multi-Release JARs?
> 
> -Alan.

Rémi


Re: Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne)

2018-03-23 Thread Remi Forax
- Mail original -
> De: "Stephen Colebourne" 
> À: "jigsaw-dev" 
> Envoyé: Vendredi 23 Mars 2018 13:51:06
> Objet: Re: Some points on JPMS/Jigsaw after he modularised some code (from 
> Stephen Colebourne)

> On 23 March 2018 at 10:19, Martijn Verburg  wrote:
>> Stephen's comments are in his blog post:
>> http://blog.joda.org/2018/03/jpms-negative-benefits.html
> 
> Firstly, I want to emphasise that my goal is effectively that of a
> retrospective, to examine what hasn't worked so well and to improve
> things from where we are. The jigsaw team did a great job in getting
> the feature out at all - I want to make sure it is used rather than
> ignored or used as a blocker to progressing the JDK.
> 
> 
>> @Stephen are you able to share the source code?
> 
> The source code is all public:
> https://github.com/ThreeTen/threeten-extra
> https://github.com/JodaOrg/joda-parent
> https://github.com/JodaOrg/joda-convert
> https://github.com/JodaOrg/joda-beans
> 
> While some of the problems are tool-based, the most fundamental issues
> are about JPMS itself.
> 
> At the heart of the problem is the split between class-path and
> module-path. Since this split has happened now, and can't "unhappen"
> what is needed is a way to make it easier to manage (fix the problems
> the split has created).
> 
> As it stands, a library developer cannot control whether they run on
> the class-path or module-path. This increases the bug-surface of the
> library, and requires testing in two different environments (which is
> not widely known). Without being able to insist that a library is on
> the module-path it is also clear that the benefits of strong
> encapsulation and reliable configuration don't apply to library
> consumers.
> 
> My belief is that a way needs to be found for a library author to
> insist that their library is run as a named module. There are probably
> a number of ways this could be achieved - I'm interested in whether
> the change makes things better, not what the specific change is.
> 
> One approach is to say that modular jar files are always treated as
> named modules:
> 
> - a modular jar on the class-path is treated as being named, not part
> of the unnamed module It is therefore encapsulated, but depends on the
> unnamed module (class-path), so does not have full reliable
> configuration. There is the potential for some incompatibility with
> this change where code that uses the modular jar now can't access the
> encapsulated packages, but this is a Good thing (as the library author
> specifically coded for that encapsulation). Any incompatibilities are
> smaller than JPMS has already caused, and could be managed with the
> existing command line flags. I would hope this does not require a JVM
> spec change to achieve.
> 
> A second approach is to say that the module author must mark modules strict:
> 
> - a module author could mark a module as "strict" in module-info.java
> so that it is not permitted to be on the class-path. This does not
> force the whole application to move to the module-path - only strict
> modules and their dependencies would need to move. At runtime, if a
> strict module is found on the class-path an error occurs. I suspect
> this requires a JVM spec change, so may be harder.
> 
> There may well be more approaches. My gut feeling is that the first
> approach is the better one, being relatively simple and implementable,
> but of course I may be wrong.
> 
> Ultimately, the requirement is that library authors who go to the
> effort of adding module-info.java should see some benefits in doing
> so, where today they only have increased cost through running as both
> an unnamed and named module.
> 
> thanks
> Stephen

You can check dynamically if a class is in the unamed module (classpath) or not 
and emit an error message.

This is exactly what the VM does when the class version is wrong.

Rémi



Re: p example (Re: Reflection: how can one access public fields (members) in a superclass ?

2018-03-13 Thread Remi Forax


- Mail original -
> De: "Rony G. Flatscher" 
> À: "Alan Bateman" , "jigsaw-dev" 
> 
> Envoyé: Mardi 13 Mars 2018 13:16:49
> Objet: Re: p example (Re: Reflection: how can one access public fields 
> (members) in a superclass ?

> Just to conclude this posting: first many thanks to Alan!
> 
> Alan's little nutshell example about how to use MethodHandles.Lookup and 
> employ
> MethodHandle s to
> invoke helped me a lot to get afloat with rewriting the reflection part for 
> the
> bindings also using
> MH. The new reflection mechanism has been implemented such that on Java 1.6/6
> and 1.7/7 it uses core
> reflection (on Java 1.7/7 MethodHandle only reflection runs in some errors 
> that
> cannot be
> circumvented, hence using core reflection) reflection with the need to do
> setAccessible(true) in
> quite a few places. The support for Java 1.8/8 and 9 can use both, core
> reflection (without a need
> for setAccessible() at all!) and MethodHandle reflection (with one case found 
> so
> far, that does not
> work on MH, but with core reflection).
> 
> The type of reflection that is being carried out is in all my use cases (my
> bridge intentionally has
> been only supporting public classes and public members) not really
> specific/relevant to jigsaw. Just
> wanted to point that out explicitly.
> 
> ---
> 
> The only grief I have at the moment is with the meaning of "public": "public"
> has become a homonym
> in jigsaw, introducing a *lot* of confusion into Java (one time "public" is
> public, another time
> "public" is not public).
> 
> This could be probably eased considerably IMHO, if there was a class Modifier
> introduced named
> "module" that would be set for reflective access to classes from non exported
> modules.

It already exists, but not at class level, 
you can open a module, or each package individually, in that case, 
setAccessible is allowed.

> 
> This would have at least the following benefits:
> 
> - communicating (and teaching!) about the publicness of a class can 
> immediately
> be clarified ("is
> the class truly public or does it reside in a non-exported module?"),

Teaching is not a big deal because you can teach package and module at the same 
time and say that it works like onions or russian dolls,
you have module access, class access and member access, by example, a public 
field is not really public if the class is not public and the package not 
exported.
If you take a look from the point of view of accessing a member (and not a 
class), things are regular at module level and class level.  

> 
> - also, programmatically it would be simple to learn whether a class object
> fetched reflectively is
> truly public or not by testing against the presence of such a "Module" 
> modifier
> bit at runtime in
> Java 9 or higher.

you can use Lookup.accessClass() [1].

> 
> ---rony

Rémi
[1] 
https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/MethodHandles.Lookup.html#accessClass-java.lang.Class-

> 
> 
> On 24.01.2018 20:48, Alan Bateman wrote:
>> On 24/01/2018 15:42, Rony G. Flatscher wrote:
>>>
>>> OK, now add to this the following class that uses p.C2 objects to access 
>>> e.g.
>>> m() via it:
>>>
>>> G:\xfer\java9modules\03-20180124-AlanBatmanP\p>type UseC2.java
>>>
>>> public class UseC2
>>> {
>>>     public static void main (String args[]) {
>>>     p.C2 o=new p.C2();
>>>     System.out.println("o="+o);
>>>     System.out.println("o.m()="+o.m());
>>>     }
>>> }
>>>
>>> Compiling all three classes works.
>>>
>>> Running "UseC2" works and outputs:
>>>
>>> G:\xfer\java9modules\03-20180124-AlanBatmanP\p>java -cp ".;.." UseC2
>>> o=p.C2@66048bfd
>>> o.m()=-1
>>>
>>> So it is possible to access m() via the p.C2 object from UseC2.
>> That's right and this is the reason for moving to the simpler example.
>>
>> To be absolutely sure then you should decompile C2.class to make sure that 
>> there
>> isn't a bridge
>> method calling C1's m2(). If you change m() to be final then that will keep 
>> the
>> bridge method from
>> complicating the picture.
>>
>> If you change UseC2 to use core reflection and you hit the issue because the
>> Method object you get
>> is p.C1.m(). Attempting to invoke this will fail with 
>> IllegalAccessException. In
>> your other mail
>> you show a code fragment where it catches exceptions and calls setAccessible 
>> -
>> I'll guess that
>> this may have been masking the issue in the Rexx bridge.
>>
>> For completeness then you may want to try out the new reflection API. I 
>> realize
>> you have to
>> compile to JDK 6 but I think you'll find it will work the same way as the
>> invokevirtual that o.m()
>> compiles to.
>>
>>     MethodHandles.Lookup lookup = MethodHandles.lookup();
>>     MethodType mt = MethodType.methodType(int.class);
>>     MethodHandle mh = lookup.findVirtual(p.C2.class, "m", mt);
>>     Object res = mh.invoke(o);
>>
> > -Alan


Re: java.beans package in java.desktop module

2018-03-07 Thread Remi Forax


- Mail original -
> De: "Stephen Colebourne" 
> À: "jigsaw-dev" 
> Envoyé: Mercredi 7 Mars 2018 13:23:10
> Objet: Re: java.beans package in java.desktop module

> What is needed is an abstraction that can work for all sorts of
> data-like classes
> 
> - classic JavaBeans
> - records
> - value types
> - HashMaps
> - JSON objects
> - etc
> 
> Its not rocket science as an API, but has been needed for many years
> as so many projects have this code duplicated (leading to lots of
> subtle differences).
> 
> The API cannot be based on method handles, as in the HashMap case
> there is no property-specific method to call.

you can bind [1] (do partial evaluation of) Map::get with the property name,

>  But there is no reason why the implementation of the interface for records 
> could not use
> method handles internally.

apart if valhalla generics are around, such interface will require to box and 
unbox values.

> 
> Stephen

Rémi

> 
> 
> On 7 March 2018 at 11:21, Remi Forax  wrote:
>> As Stephen said, with the introduction of the Pattern Matching in the near
>> future, an API to extract the values from an object (the de-constructor API) 
>> or
>> at least from a record object will have to be created, but it may be based on
>> method handles, so perhaps not using a direct interface.
>>
>> Rémi
>>
>> - Mail original -
>>> De: "Guillaume Smet" 
>>> À: "Stephen Colebourne" 
>>> Cc: "jigsaw-dev" 
>>> Envoyé: Mardi 6 Mars 2018 18:45:21
>>> Objet: Re: java.beans package in java.desktop module
>>
>>> Hi Stephen,
>>>
>>> On Tue, Mar 6, 2018 at 3:29 PM, Stephen Colebourne 
>>> wrote:
>>>
>>>> It had been my hope that we might see a replacement for the java.beans
>>>> package. I drew up a rough prototype here:
>>>>  https://github.com/jodastephen/property-alliance
>>>> based on previous work in Joda-Beans. More info here:
>>>>  http://jodastephen.github.io/property-alliance/
>>>>
>>>> If an interface-based design were adopted, it could be implemented by
>>>> the existing JavaBeans code and also by future language changes such
>>>> as records (data classes), while the API could be used far more
>>>> broadly, such as in Hibernate or EL.
>>>>
>>>> Sadly, I haven't had the time or energy to progress this, but the need is
>>>> there.
>>>
>>>
>>> Thanks for sharing. It indeed sound like something worth pursuing.
>>>
>>> Having a new API in the JDK indeed seems the only way out for the
>>> java.beans issue.
>>>
>>> --
> >> Guillaume


Re: java.beans package in java.desktop module

2018-03-07 Thread Remi Forax
As Stephen said, with the introduction of the Pattern Matching in the near 
future, an API to extract the values from an object (the de-constructor API) or 
at least from a record object will have to be created, but it may be based on 
method handles, so perhaps not using a direct interface.

Rémi

- Mail original -
> De: "Guillaume Smet" 
> À: "Stephen Colebourne" 
> Cc: "jigsaw-dev" 
> Envoyé: Mardi 6 Mars 2018 18:45:21
> Objet: Re: java.beans package in java.desktop module

> Hi Stephen,
> 
> On Tue, Mar 6, 2018 at 3:29 PM, Stephen Colebourne 
> wrote:
> 
>> It had been my hope that we might see a replacement for the java.beans
>> package. I drew up a rough prototype here:
>>  https://github.com/jodastephen/property-alliance
>> based on previous work in Joda-Beans. More info here:
>>  http://jodastephen.github.io/property-alliance/
>>
>> If an interface-based design were adopted, it could be implemented by
>> the existing JavaBeans code and also by future language changes such
>> as records (data classes), while the API could be used far more
>> broadly, such as in Hibernate or EL.
>>
>> Sadly, I haven't had the time or energy to progress this, but the need is
>> there.
> 
> 
> Thanks for sharing. It indeed sound like something worth pursuing.
> 
> Having a new API in the JDK indeed seems the only way out for the
> java.beans issue.
> 
> --
> Guillaume


Re: Finding module-info.class without loading a jar

2018-02-24 Thread Remi Forax


On February 24, 2018 12:33:25 PM UTC, Mark Raynsford  
wrote:
>On 2018-02-24T07:53:16 +
>Alan Bateman  wrote:
>>
>> The JarFile API does this for you. Are you sure you've used the 
>> constructor that specifies the runtime version? Once you do that then
>
>> getJarEntry("module-info.class") will locate the module-info.class in
>
>> the versioned section. Use JarEntry::getRealName to satisfy yourself 
>> that it always locates the right one (say where you have a 
>> module-info.class in the top-level directory and one each in 
>> META-INF/versions/9 and META-INF/versions/10).
>
>Ah, thank you. I had completely overlooked this constructor.
>
>> ModuleDescriptor.read is an easy way to parse the module-info.class
>in 
>> case you need it.
>
>Yes, I'm planning to move to this from ASM.

The main difference between the ModuleDescriptor and the ModuleVisitor of ASM 
is that you can read the annotations with the ModuleVisitor.
So it may not worth having ASM as dependency.

Remi


-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.


OpenJDK 10 and Oracle JDK10 doesn't have the same default modules

2018-02-04 Thread Remi Forax
Hi all,
it seems that the OpenJDK 10 and OracleJDK 10 doest not declare the same set of 
default modules, so java --add-modules ALL-DEFAULT do not behave the same way :(

With Oracle JDK 10 b42, module java.scripting is part of the default modules
https://travis-ci.org/sormuras/beautiful_logger/jobs/337153634

With Oracle JDK 10 b42, module java.scripting is NOT part of the default 
modules,
so javax/script/ScriptException is not found
https://travis-ci.org/sormuras/beautiful_logger/jobs/337153635

Should not ALL-DEFAULT mean the same set of default modules for a JDK release ?

cheers,
Rémi



Re: Where do modules go in jlink images?

2018-01-07 Thread forax
- Mail original -
> De: "Nicolai Parlog" 
> À: fo...@univ-mlv.fr
> Cc: "jigsaw-dev" 
> Envoyé: Dimanche 7 Janvier 2018 14:26:08
> Objet: Re: Where do modules go in jlink images?

> Hi Remi,
> 
> but the exact same thing happens when the JDK is linked with jlink,
> right? I just now see that the native binaries from the jmod archives
> are also present in lib/.
> 
> So does that really mean that the jmods folder plays no role for the
> running JVM? 

yes

> What is it there for?

make jlink working :)
the transformation done by jlink is not reversible.

> 
> so long ... Nicolai

Rémi

> 
> 
> 
> On 07.01.2018 14:15, Remi Forax wrote:
>> All classes are merged in the file named 'modules' and the native
>> libraries are extracted from the jmods and put in the lib folder,
>> because jlink create an image for a specific architecture/OS.
>> 
>> cheers, Rémi
>> 
>> - Mail original -
>>> De: "Nicolai Parlog"  À: "jigsaw-dev"
>>>  Envoyé: Dimanche 7 Janvier 2018
>>> 13:57:51 Objet: Where do modules go in jlink images?
>> 
>>> Hi!
>>> 
>>> Another question about jlink (guess what chapter I'm working on
>>> ;) ). I've tried to find out where jlink puts the content of the
>>> jmods folder, but hit a wall.
>>> 
>>> I created a full runtime image with jlink and diffed the file
>>> trees (including disk usage), but all that got me was that my
>>> image doesn't contain bin/jmc, jmods, lib/misssioncontrol,
>>> lib/src.zip, and lib/plugin-legacy.jar - all of that is expected,
>>> but where are the class files from the jmod archives?
>>> 
>>> Now I wonder, whether these are just ornamental - does
>>> lib/modules contain all the bytecode? I'm confused.
>>> 
>>> so long ... Nicolai
>>> 
>>> 
>>> 
>>> --
>>> 
>>> PGP Key:
>>> http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
>>>
>>>
>>> 
> Web:
>>> http://codefx.org a blog about software development
>>> https://www.sitepoint.com/java high-quality Java/JVM content
>>> http://do-foss.de Free and Open Source Software for the City of
>>> Dortmund
>>> 
>>> Twitter: https://twitter.com/nipafx
>> 
> 
> --
> 
> PGP Key:
>http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
> 
> Web:
>http://codefx.org
>a blog about software development
>https://www.sitepoint.com/java
>high-quality Java/JVM content
>http://do-foss.de
>Free and Open Source Software for the City of Dortmund
> 
> Twitter:
> https://twitter.com/nipafx


Re: Where do modules go in jlink images?

2018-01-07 Thread Remi Forax
All classes are merged in the file named 'modules' and the native libraries are 
extracted from the jmods and put in the lib folder, because jlink create an 
image for a specific architecture/OS.

cheers,
Rémi

- Mail original -
> De: "Nicolai Parlog" 
> À: "jigsaw-dev" 
> Envoyé: Dimanche 7 Janvier 2018 13:57:51
> Objet: Where do modules go in jlink images?

> Hi!
> 
> Another question about jlink (guess what chapter I'm working on ;) ).
> I've tried to find out where jlink puts the content of the jmods
> folder, but hit a wall.
> 
> I created a full runtime image with jlink and diffed the file trees
> (including disk usage), but all that got me was that my image doesn't
> contain bin/jmc, jmods, lib/misssioncontrol, lib/src.zip, and
> lib/plugin-legacy.jar - all of that is expected, but where are the
> class files from the jmod archives?
> 
> Now I wonder, whether these are just ornamental - does lib/modules
> contain all the bytecode? I'm confused.
> 
> so long ... Nicolai
> 
> 
> 
> --
> 
> PGP Key:
>http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
> 
> Web:
>http://codefx.org
>a blog about software development
>https://www.sitepoint.com/java
>high-quality Java/JVM content
>http://do-foss.de
>Free and Open Source Software for the City of Dortmund
> 
> Twitter:
> https://twitter.com/nipafx


Re: Update a runtime image

2017-12-31 Thread Remi Forax
jlink is not a distribution tool,
you need to use a tool on top of jlink, in the past, i've used the rpm format 
for that.

We have stopped to distribute deltas when we have moved to docker that as far 
as i know as no concept of incremental update.

Rémi 

- Mail original -
> De: "Alan Bateman" 
> À: "Thomas Brand" , "jigsaw-dev" 
> 
> Envoyé: Dimanche 31 Décembre 2017 16:55:37
> Objet: Re: Update a runtime image

> On 31/12/2017 15:42, Thomas Brand wrote:
>> Hello, I have a question related to jlink.
>>
>> In a scenario where a runnable image created with jlink is installed, and
>> it should be updated, what are good strategies for doing this?
>>
>> I think if an update will introduce new dependencies this will be
>> different to updates that are known to be satisfied with what is already
>> in the image.
>>
>> The idea is to 'ship' just a small sort of diff / replacement part that
>> can be applied to the 'old' image to become 'new'.
>>
> The jlink tool always generates a new run-time image, there is no
> support at this time to install additional modules into a run-time image.
> 
> -Alan.


Re: Module system and services directory in META-INF

2017-12-27 Thread Remi Forax
I believe that the doc has been moved to the doc of the ServiceLoader
  https://docs.oracle.com/javase/9/docs/api/java/util/ServiceLoader.html

Rémi

- Mail original -
> De: "David Holmes" 
> À: "Jochen Theodorou" , "jigsaw-dev" 
> 
> Envoyé: Mercredi 27 Décembre 2017 07:27:21
> Objet: Re: Module system and services directory in META-INF

> On 27/12/2017 5:43 AM, Jochen Theodorou wrote:
>> On 26.12.2017 19:30, Jochen Theodorou wrote:
>>> hi all,
>>>
>>> Do all files in META-INF/services now have to be well formed according
>>> to the SPI strucutres in Java, or is it still valid to have other
>>> files in there as well? Before the module system it was no problem to
>>> have in there service decriptors, that are services, but do not use
>>> the ServiceProvider structure. Nothing was disturbed as long as nobody
>>> tried to load it that way. Now this seems to be different and I am
>>> wondering where this got specified.
>>>
>>> Because the spec does not try to define file layouts so much I have
>>> the feeling that this is not really specified by the java module
>>> system. Was it then a decision by some maven plugin writers or maybe
>>> even the javac guys?
>> 
>> I found
>> https://docs.oracle.com/javase/8/docs/technotes/guides/jar/jar.html#Service_Provider
>> 
>> Is there no Java 9 version? Also the spec is not specific enough for
>> this as it sounds more like implicit assumption than specification
> 
> https://docs.oracle.com/javase/9/docs/specs/jar/jar.html
> 
> but the Service provider section is gone. Also the document seems
> mis-formatted.
> 
> David
> 
>> bye Jochen


Re: Resolution exception when service interface gets exposed via --add-exports

2017-11-22 Thread forax
My opinion is that we should fix --patch-module to allow to patch the 
module-info.class, 
you always end up with this restriction once you use --patch-module. 

Rémi 

> De: "Gunnar Morling" 
> À: "Alan Bateman" 
> Cc: "Remi Forax" , "jigsaw-dev" 
> 
> Envoyé: Mercredi 22 Novembre 2017 12:49:10
> Objet: Re: Resolution exception when service interface gets exposed via
> --add-exports

> Just in case others end up in the same situation: --patch-module (alone) 
> doesn't
> do the trick as it can't be used to override the module-info.class of the
> jdk.jlink module. So I'm just using it to inject my plug-in and then use a 
> Java
> Agent to register that one as a service provider (via
> Instrumentation#redefineModule()).
> If I find the time I'll try and blog about it; needless to say, that a public,
> supported API would be much appreciated :) Thanks Alan and Remi for your help!

> 2017-11-21 11:46 GMT+01:00 Alan Bateman < [ mailto:alan.bate...@oracle.com |
> alan.bate...@oracle.com ] > :

>> On 21/11/2017 09:29, Remi Forax wrote:

>>> Hi Alan,
>>> I use jlink plugins for 2 things,
>>> - code generation, i.e. find patterns in the bytecode and optimize them 
>>> because
>>> at link time you are in a closed world, so something that was dynamic is not
>>> anymore.
>>> - just bytecode crawling to find specific annotations to improve startup.

>>> I believe implementation like Weld or Spring should have access to an open 
>>> API
>>> to be able to do thing kind of optimizations, imagine, you could create all
>>> proxies upfront or better use invokedynamic to even avoid proxy creation at
>>> all.

>>> The plugin API doesn't have to be complex, it's a read/replace/append of
>>> bytecode files.

>> Good work was done in JDK 9 to get the plugin API to its current state. We 
>> had
>> hoped to expose it in an incubating module but it didn't happen as it would
>> have required splitting the jdk.jlink module in an awkward way. Yes, it 
>> should
>> be revisited.
>> -Alan


Re: Resolution exception when service interface gets exposed via --add-exports

2017-11-21 Thread Remi Forax
Hi Alan,
I use jlink plugins for 2 things,
- code generation, i.e. find patterns in the bytecode and optimize them because 
at link time you are in a closed world, so something that was dynamic is not 
anymore.
- just bytecode crawling to find specific annotations to improve startup.

I believe implementation like Weld or Spring should have access to an open API 
to be able to do thing kind of optimizations, imagine, you could create all 
proxies upfront or better use invokedynamic to even avoid proxy creation at all.

The plugin API doesn't have to be complex, it's a read/replace/append of 
bytecode files.

Rémi

- Mail original -
> De: "Alan Bateman" 
> À: "Gunnar Morling" 
> Cc: "jigsaw-dev" 
> Envoyé: Mardi 21 Novembre 2017 09:33:33
> Objet: Re: Resolution exception when service interface gets exposed via 
> --add-exports

> On 20/11/2017 21:56, Gunnar Morling wrote:
>> I see, thanks for the clarification.
>>
>> It's a pity, though, I hoped to employ that approach for providing a
>> custom jlink plug-in (implementation of
>> jdk.tools.jlink.plugin.Plugin). So it seems that's not possible until
>> jdk.jlink exports j.t.j.plugin? Are there any plans to do so some time
>> soon?
>>
>> Even if it was marked as experimental/incubating, being able to
>> provide custom jlink plug-ins would enable interesting use cases.
>>
> There was interest in creating an incubating module in JDK 9 but it was
> put aside due to the issues of tool modules depending on an incubating
> module. The other issue is that most of the interesting plugins to date
> do code generation at link time and are deeply tied to specific areas of
> java.base and other core modules (they can't really live outside of the
> jdk repo). So while it might be desirable to expose the interface for
> plugins, I think it will need further exploration to see if it makes
> sense or not.
> 
> -Alan


Re: Resolution exception when service interface gets exposed via --add-exports

2017-11-20 Thread Remi Forax
Hi Gunnar,
you can use --patch-module to add your plugin inside jdk.jlink if you want to 
experiment.

Rémi

- Mail original -
> De: "Gunnar Morling" 
> À: "Alan Bateman" 
> Cc: "jigsaw-dev" 
> Envoyé: Lundi 20 Novembre 2017 22:56:35
> Objet: Re: Resolution exception when service interface gets exposed via 
> --add-exports

> I see, thanks for the clarification.
> 
> It's a pity, though, I hoped to employ that approach for providing a custom
> jlink plug-in (implementation of jdk.tools.jlink.plugin.Plugin). So it
> seems that's not possible until jdk.jlink exports j.t.j.plugin? Are there
> any plans to do so some time soon?
> 
> Even if it was marked as experimental/incubating, being able to provide
> custom jlink plug-ins would enable interesting use cases.
> 
> 2017-11-20 22:24 GMT+01:00 Alan Bateman :
> 
>> On 20/11/2017 21:04, Gunnar Morling wrote:
>>
>>> Hi,
>>>
>>> I'm having one module which defines a service interface but does not
>>> export
>>> that interface's package. In another module I wish to create an
>>> implementation of that service, which should be possible via
>>> --add-exports.
>>>
>>> I can successfully compile that second module, but running it fails:
>>>
>>>  Error occurred during initialization of boot layer
>>>  j.l.m.ResolutionException: Module com.example.b does not read a
>>> module
>>> that exports com.example.a
>>>
>>> It seems as if the service binding happens before the --add-exports of the
>>> java invocation is processed. Please see below for the steps to reproduce.
>>>
>>> Is it a bug or am I trying to do something unsupported here?
>>>
>>> At run-time, the --add-reads/--add-exports options are the equivalent of
>> the module invoking addReads or addExports. So yes, they come into effect
>> after resolution and service binding (and so after the resolution
>> consistency check that detects that a module uses a service type in a
>> package that isn't exported by anyone to the module).
>>
>> -Alan


Re: JDK-8189251

2017-10-14 Thread Remi Forax
Hi Gili,
the main issue is that an automatic module can see classes from the classpath, 
but it also exports all its package so there is no encapsulation, and once you 
require one automatic module all automatic modules from the module path are 
visible.

So an automatic module is a great tool when you transitioned to the module 
world, but in fine, you do not want any automatic modules in you dependency 
graph.

regards,
Rémi

- Mail original -
> De: "cowwoc" 
> À: "jigsaw-dev" 
> Envoyé: Samedi 14 Octobre 2017 04:13:31
> Objet: JDK-8189251

> Hi,
> 
> I'm trying to understand the reasoning behind
> https://bugs.openjdk.java.net/browse/JDK-8189251 being closed as Won't Fix.
> 
> Can someone please clarify the downside of named automatic-modules vs normal
> modules? I assume that there is some sort of downside; otherwise, we
> wouldn't be warning about them.
> 
> Thank you,
> Gili
> 
> 
> 
> --
> Sent from: http://jigsaw-dev.1059479.n5.nabble.com/


Re: javax.inject module?

2017-10-04 Thread forax
> De: "Lance Andersen" 
> À: "Remi Forax" 
> Cc: "Tom De Wolf" , "jigsaw-dev"
> 
> Envoyé: Mercredi 4 Octobre 2017 20:25:41
> Objet: Re: javax.inject module?

> Hi Remi,
> Sorry if my message was confusing.

it's not, it's my mind which is confused :) 

> JEP 200, [ http://openjdk.java.net/jeps/200 | 
> http://openjdk.java.net/jeps/200 ]
> , defines that:

>> ——
>> Standard modules, whose specifications are governed by the JCP, have names
>> starting with the string "java." .
>> -

> JSR 330 specifies the javax.inject package. The suggested module name would be
> java.inject.

> The hope is that the spec leads for JSR 330, will at a minimum update their 
> API
> jar to include the MANIFEST attribute Automatic-Module-Name
> Automatic-Module-Name: java.inject

> This will help developers in the migration to modules.

thanks for the answer, 
Rémi 

> Best
> Lance

> On Oct 4, 2017, at 2:13 PM, Remi Forax < [ mailto:fo...@univ-mlv.fr |
> fo...@univ-mlv.fr ] > wrote:

>> Hi Lance,
>> i'm not sure you have the right to use java or jdk as prefix for your 
>> module, or
>> do you mean "javax.inject" ?

>> Rémi

>> - Mail original -

>>> De: "Lance Andersen" < [ mailto:lance.ander...@oracle.com |
>>> lance.ander...@oracle.com ] >
>>> À: "Tom De Wolf" < [ mailto:tom.dew...@aca-it.be | tom.dew...@aca-it.be ] >
>>> Cc: "jigsaw-dev" < [ mailto:jigsaw-dev@openjdk.java.net |
>>> jigsaw-dev@openjdk.java.net ] >
>>> Envoyé: Mercredi 4 Octobre 2017 18:37:53
>>> Objet: Re: javax.inject module?

>>> You can grab the jar from maven central
>>> [ https://mvnrepository.com/artifact/javax.inject/javax.inject/1 |
>>> https://mvnrepository.com/artifact/javax.inject/javax.inject/1 ]

>>> The JSR can be found here https://jcp.org/en/jsr/detail?id=330.

>>> I am hoping to try and get the JSR 330 spec leads to support updating the
>>> implementation jar with an automatic module name if at all possible. The
>>> proposed module name is java.inject

>>> Best
>>> Lance
>>> On Oct 4, 2017, at 11:17 AM, Tom De Wolf  wrote:

>>>> Hi,

>>>> Does someone know why the javax.inject annotations contained in for example
>>>> the following maven dependency are not part of the jdk?

>>>> 
>>>> 
>>>> javax.inject
>>>> javax.inject
>>>> 1
>>>> 

>>>> Who or which dependency/jar can be used to depend on this set of injection
>>>> annotations as an explicit module or a module with a reserved automatic
>>>> module name? Who is responsible to tackle this for javax.inject?

>>>> Thanks,

>>>> Tom De Wolf
>>>> Architect
>>>> ACA-IT Solutions
>>>> Belgium

>>> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
>>> Oracle Java Engineering
>>> 1 Network Drive
>>> Burlington, MA 01803
>>> lance.ander...@oracle.com

> [ http://oracle.com/us/design/oracle-email-sig-198324.gif | Lance Andersen|
> Principal Member of Technical Staff | +1.781.442.2037Oracle Java Engineering 1
> Network Drive Burlington, MA 01803 ] [ mailto:lance.ander...@oracle.com |
> lance.ander...@oracle.com ]


Re: javax.inject module?

2017-10-04 Thread Remi Forax
Hi Lance,
i'm not sure you have the right to use java or jdk as prefix for your module, 
or do you mean "javax.inject" ?

Rémi

- Mail original -
> De: "Lance Andersen" 
> À: "Tom De Wolf" 
> Cc: "jigsaw-dev" 
> Envoyé: Mercredi 4 Octobre 2017 18:37:53
> Objet: Re: javax.inject module?

> You can grab the jar from maven central
> https://mvnrepository.com/artifact/javax.inject/javax.inject/1
> 
> The JSR  can be found here https://jcp.org/en/jsr/detail?id=330.
> 
> I am hoping to try and get the JSR 330 spec leads to support updating the
> implementation jar with an automatic module name if at all possible. The
> proposed module name is java.inject
> 
> Best
> Lance
> On Oct 4, 2017, at 11:17 AM, Tom De Wolf  wrote:
> 
>> Hi,
>> 
>> Does someone know why the javax.inject annotations contained in for example
>> the following maven dependency are not part of the jdk?
>> 
>> 
>> 
>>javax.inject
>>javax.inject
>>1
>> 
>> 
>> Who or which dependency/jar can be used to depend on this set of injection
>> annotations as an explicit module or a module with a reserved automatic
>> module name? Who is responsible to tackle this for javax.inject?
>> 
>> Thanks,
>> 
>> Tom De Wolf
>> Architect
>> ACA-IT Solutions
>> Belgium
> 
> 
> 
> 
> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
> Oracle Java Engineering
> 1 Network Drive
> Burlington, MA 01803
> lance.ander...@oracle.com


Re: issue deriving automatic module name

2017-10-04 Thread Remi Forax
Hi Paul,
the whole idea of jigsaw is that if you pass the startup check of the module 
graph, there will be not surprise anymore,
so preventing the VM to boot is by design.

Here, you will not be able to reference the jar in another module-info so there 
is no point to start with a wrong configuration.

Rémi

- Mail original -
> De: "Paul Bakker" 
> À: "jigsaw-dev" 
> Envoyé: Mercredi 4 Octobre 2017 02:44:01
> Objet: issue deriving automatic module name

> Hi,
> 
> I ran into an issue testing some migration scenarios. When using
> jboss-transaction-api_1.2_spec-1.0.1.Final.jar as an automatic module, startup
> fails with:
> 
> Caused by: java.lang.IllegalArgumentException: jboss.transaction.api.1.2.spec:
> Invalid module name: '1' is not a Java identifier
> 
> This is clearly a strange JAR name, and can be fixed by providing a module 
> name,
> but it should probably not prevent startup.
> 
> Best,
> 
> Paul


Re: Compile Jigsaw project with Intellij IDEA supported structure

2017-09-27 Thread Remi Forax
And you can use the same trick with Eclipse too that said the Eclipse compiler 
does not support the multi-module layout (only one module by project).

Rémi


On September 27, 2017 2:09:06 PM GMT+02:00, Trisha Gee  
wrote:
>Hi Alexandru,
>
>The structure in IntelliJ IDEA isn't as strict as that, it doesn't
>require
>you to have a "src" folder.  For example you could have:
>
>com.myModule.m1
>/module-info.java
>/com.myModule.m2
>/Main.java
>
>com.myModule.m2
>/module-info.java
>/com.myModule.m2
>/Main.java
>
>You just need to mark com.myModule.m1 and com.myModule.m2 as your
>sources
>root instead of having a src folder as the sources root.
>
>https://www.jetbrains.com/help/idea/configuring-content-roots.html#d85322e277
>
>Trisha
>
>-- Forwarded message -
>Date: Tue, 26 Sep 2017 15:37:18 +0200
>From: "Alexandru Jecan" 
>To: 
>Subject: Compile Jigsaw project with Intellij IDEA supported structure
>Message-ID: <000e01d336cc$93a52ba0$baef82e0$@yahoo.com>
>Content-Type: text/plain;   charset="utf-8"
>
>Hello,
>
>
>
>In order to make my Jigsaw project be supported by Intellij IDEA
>2017.2, I
>need a structure like this, according to IDEA Video:
>https://www.youtube.com/watch?v=WL48zkLvK3I (00:10):
>
>
>
>com.myModule.m1
>
>/src
>
>  /module-info.java
>
> /com.myModule.m2
>
> /Main.java
>
>
>
>com.myModule.m2
>
>/src
>
>  /module-info.java
>
> /com.myModule.m2
>
> /Main.java
>
>
>
>
>
>How do I perform multi-module compilation in this case using the
>--module-source-path option?
>
>
>
>As far as I know, to perform multi-module compilation, the source code
>directory should have the same name like the name of the module.
>
>In this case, I have the ?src? folder there which does not comply to
>this
>rule.
>
>
>
>
>
>Best regards
>
>
>
>Alexandru Jecan

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.


Re: Scanning multi version jars?

2017-09-19 Thread Remi Forax
Hi Greg,
the notion of inner classes do not exist at runtime (apart if you explicitly 
ask by reflection).

The compiler desugar inner classes to several classes, so the compiler needs 
attributes (InnerClasses and EnclosingMethod) in the classfile to be able to 
reconstruct the java source file view of the world when it sees already 
compiled classes.

But the VM/runtime doesn't need those attributes, the VM doesn't care about 
inner classes. There is a comment at the end of section 4.7.6 of the JVMS that 
explain that the VM does not check the InnerClasses attribute at runtime 
"Oracle's Java Virtual Machine implementation does not check the consistency of 
an InnerClasses attribute against a class file representing a class or 
interface referenced by the attribute." 

So to answer to your question, the classloader do not care about inner classes.

Note that in a close future with the introduction of nestmates [1], it will be 
another story, but we have at least to wait 18.9 for that.

Rémi
[1] http://openjdk.java.net/jeps/181

- Mail original -
> De: "Greg Wilkins" 
> À: "Stephen Felts" 
> Cc: "jigsaw-dev" , "core-libs-dev" 
> 
> Envoyé: Mardi 19 Septembre 2017 06:37:37
> Objet: Re: Scanning multi version jars?

> Stephen,
> 
> I think the use-case can be pretty well defined.
> 
> There should be an enumeration/iterator/stream available that provides the
> contents of a jar file as it would be seen/interpreted by the JVMs
> classloader.So if the classloader is doing any processing to handle
> versioned classes/resources, then we need an iterator that implements the
> exact same logic.
> 
> Which raises an interesting point   with the multi versioned jar I have
> used as an example, which contains:
> 
>   - org/example/Foo.class
>   - org/example/Foo$Bar.class
>   - META-INF/versions/9/org/example/Foo.class
> 
> What does the classloader do when asked to load "org.example.Foo$Bar" ?
> If it loads it OK, then the JarFile enumerator/iterator/stream should also
> return it.   If it throws a ClassNotFoundException, then the
> JarFile enumerator/iterator/stream should skip it.
> 
> Currently the classloader will happily return a resource for a base inner
> class even if its outerclass does not refer to it, so that suggests that
> the iteration should also not process out the inappropriate inner classes.
> However I think it could be argued that the loader should not load it.
> 
> Eitherway, there should be an iteration available that is entirely
> consistent with what the classloader does.
> 
> regards
> 
> 
> 
> 
> On 19 September 2017 at 13:41, Stephen Felts 
> wrote:
> 
>> Thanks for the clarification – I overstated the “any JarEntry”.
>>
>> I didn’t look at VersionedStream so I now understand the limitations you
>> mention.
>>
>>
>>
>> In my case, it’s necessary to look at all files in the jar file to do the
>> elimination of unneeded ordinary/inner classes so JarInputStream
>> getNextJarEntry()
>> can be used.
>>
>> By using the versioned JarFile constructor, getting the JarEntry returns
>> the right one for processing.  If I needed further filtering on the file
>> names, I’d need to return the real file names.
>>
>>
>>
>> Maybe the use case isn’t so universal or well defined.
>>
>>
>>
>>
>>
>>
>>
>>
>> *From:* Greg Wilkins [mailto:gr...@webtide.com]
>> *Sent:* Monday, September 18, 2017 9:33 PM
>> *To:* Stephen Felts 
>> *Cc:* Paul Sandoz ; jigsaw-dev <
>> jigsaw-dev@openjdk.java.net>; core-libs-...@openjdk.java.net
>>
>> *Subject:* Re: Scanning multi version jars?
>>
>>
>>
>> Stephen,
>>
>>
>>
>> It is not the case that the getName() always returns the path starting
>> with "META-INF/versions/". Specifically, if the entry is obtained from
>> getJarEntry() API (and not from the enumerator), then the name is that of
>> the unversioned file, but the metadata and contents obtained using the
>> jarEntry are for the versioned entry.
>>
>>
>>
>> For example the following code:
>>
>>
>>
>> JarFile jarFile = new JarFile(new File("/tmp/example.jar"),
>>   false,
>>   JarFile.OPEN_READ,
>>   Runtime.version());
>> JarEntry entry = jarFile.getJarEntry("org/example/OnlyIn9.class");
>> System.err.printf("%s -> %s%n",entry.getName(),IO.toString(jarFile.
>> getInputStream(entry)));
>>
>> when run against a jar where the class files contain just the text of
>> their full path produces the following output:
>>
>>
>>
>> org/example/OnlyIn9.class -> META-INF/versions/9/org/example/OnlyIn9.class
>>
>>
>>
>> There is nothing in the public API of the JarEntry so obtained that
>> indicates that it the versioned entry, nor can I distinguish it from an
>> entry obtained by iteration that may report the same name (if the entry was
>> also in the base), although at least equals does return false.
>>
>>
>>
>> Moreover, the proposed stream API as represented by the current
>> implementation of jdk.internal.util.jar.VersionedStream, a

Re: Moving to Java 9 - module-info.class not found for module

2017-09-02 Thread forax
> De: "Rahman USTA" 
> À: "Remi Forax" 
> Cc: "Alex Buckley" , "jigsaw-dev"
> 
> Envoyé: Samedi 2 Septembre 2017 13:30:24
> Objet: Re: Moving to Java 9 - module-info.class not found for 
> module

> Thank you, Remi and Alex.
> I think automatic modules can be supported by jlink. It could be a transition
> feature like --illegal-access imho.

The whole idea of jlink is that unlike classical Java it works under the 
assumption that the world is closed i.e. all the bytecodes that will be loaded 
is known, so it can do "whole world" optimizations like i know all the lambda 
forms that will be used by the string concatenation because i know all the call 
sites that does string concatenation so i can pre-generate them. 
If you have an automatic module, it can have dependencies on jars from the 
classpath, the close world assumption does not hold. 

If you are in a hacky mood, there is a simple solution to transform an 
automatic module to an explicit one, create a module-info.java, compile it and 
update the jar of the automatic module with the module-info.class or better ask 
the maintainer of the jar to do the work for you :) 

> Kind regards.

regards, 
Rémi 

> 2017-09-02 0:59 GMT+03:00 Remi Forax < [ mailto:fo...@univ-mlv.fr |
> fo...@univ-mlv.fr ] > :

>> As Alex said,
>> jackson.databind is not an explicit module.

>> You should report that issue to their bugtracker.

>> cheers,
>> Rémi

>> - Mail original -
>>> De: "Alex Buckley" < [ mailto:alex.buck...@oracle.com | 
>>> alex.buck...@oracle.com
>> > ] >
>>> À: "jigsaw-dev" < [ mailto:jigsaw-dev@openjdk.java.net |
>> > jigsaw-dev@openjdk.java.net ] >
>> > Envoyé: Vendredi 1 Septembre 2017 23:06:16
>>> Objet: Re: Moving to Java 9 - module-info.class not found for 
>> > module

>> > On 9/1/2017 1:21 PM, Rahman USTA wrote:
>> >> java --module-path
>> >> %JAVA_HOME%/jmods;target\terminalfx.jar;target\dependency --add-modules
>> >> terminalfx -m terminalfx/com.terminalfx.AppStarter

>> > (You shouldn't need the --add-modules, since terminalfx is already the
>> > main module.)

>> >> It works normally. Then, I want to generate a jlink image with the
>> >> following script

>> >> jlink --module-path
>> >> %JAVA_HOME%/jmods;target\terminalfx.jar;target\dependency --add-modules
>> >> terminalfx --launcher terminalfx=terminalfx/com.terminalfx.AppStarter
>> >> --output target/release

>> >> However it gives me the following error;

>> >> Error: module-info.class not found for jackson.databind module

>> > I suspect jackson.databind is an automatic module. jlink does not
>> > support linking of automatic modules because they can rely on the
>> > arbitrary content of the classpath, which goes against the idea of a
>> > self-contained Java runtime.

>> > Alex

> --
> Rahman USTA
> Istanbul JUG
> [ https://github.com/rahmanusta | https://github.com/rahmanusta ]


Re: Moving to Java 9 - module-info.class not found for module

2017-09-01 Thread Remi Forax
As Alex said,
jackson.databind is not an explicit module.

You should report that issue to their bugtracker.

cheers,
Rémi

- Mail original -
> De: "Alex Buckley" 
> À: "jigsaw-dev" 
> Envoyé: Vendredi 1 Septembre 2017 23:06:16
> Objet: Re: Moving to Java 9 - module-info.class not found for  
> module

> On 9/1/2017 1:21 PM, Rahman USTA wrote:
>> java --module-path
>> %JAVA_HOME%/jmods;target\terminalfx.jar;target\dependency --add-modules
>> terminalfx -m terminalfx/com.terminalfx.AppStarter
> 
> (You shouldn't need the --add-modules, since terminalfx is already the
> main module.)
> 
>> It works normally. Then, I want to generate a jlink image with the
>> following script
>>
>> jlink --module-path
>> %JAVA_HOME%/jmods;target\terminalfx.jar;target\dependency --add-modules
>> terminalfx --launcher terminalfx=terminalfx/com.terminalfx.AppStarter
>> --output target/release
>>
>> However it gives me the following error;
>>
>> Error: module-info.class not found for jackson.databind module
> 
> I suspect jackson.databind is an automatic module. jlink does not
> support linking of automatic modules because they can rely on the
> arbitrary content of the classpath, which goes against the idea of a
> self-contained Java runtime.
> 
> Alex


Re: Jigsaw and containers

2017-08-31 Thread Remi Forax
Hi Daniel,
several points first,
the JRE doesn't exist anymore apart for compatibility (the jre is just a bunch 
of modules) and a jar can be a modular jar, just have a module-info.class in 
the jar, so applications that uses jars can be modular, they do not have to use 
jlink.

So you can have a jlink image as small as just java.base and all your 
applications and their dependencies as modular jars, the modules will be shared 
by the different applications, or a jlink all applications and have no sharing 
at all and all the states in between.

Rémi

- Mail original -
> De: "Daniel Latrémolière" 
> À: "Alan Bateman" , "jigsaw-dev" 
> 
> Envoyé: Jeudi 31 Août 2017 20:47:35
> Objet: Re: Jigsaw and containers

>> Are you using `jlink` to create a run-time image for each application?
>> (just trying to establish if you are actually running into an issue or
>> not).
> With jlink per application, all will be working without problems, but
> each application will have his own copy of JRE and common libraries (big
> size).
> 
> With jlink only for jre and common libraries, these parts can be shared
> between applications. But specific parts of each application will be
> only in JAR files (and not be modularized).
> 
> These two choices works as reasonable targets, but no one is really
> completely good (big size or no full modularization).
> 
> Daniel.


Re: Jigsaw questions

2017-07-13 Thread Remi Forax
- Mail original -
> De: "Oleg Tsal-Tsalko" 
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Jeudi 13 Juillet 2017 09:34:53
> Objet: Jigsaw questions

> Dear experts,
> 
> I'm from JUG UA and currently playing with new Java 9 Module System using
> Early Access Jigsaw build #174. As an exercise I'm modularising JUnit 5. In
> process I faced couple of issues and have couple of questions to you:
> 
>   1. *How to compile test classes that are packaged in same packages as
>   production code? *When I'm trying to compile test sources separately
>   from application sources that have been already modularised (packaged in
>   modules) I'm getting errors saying that particular packages already found
>   in certain modules on module path. What are recommendations (best practice)
>   here?

--patch-module is your friend.

The other solution is to put the test classes in another module and to be able 
to merge module (the tested module and the test module) but it requires support 
either by the build tool you use or by JUnit itself.


>   2. When I'm trying to run unit tests placed on classpath using
>   modularised JUnit5 library put on module path I'm getting errors
> like "*Could
>   not load class with
>   name: org.junit.platform.commons.util.CollectionUtilsTests*". I have
>   deduced that it is because CollectionUtilsTests class lives in same
>   package as already exists in junit.platform.commons module placed on
>   module path. If I run org.junit.mytests.SimpleTest from custom unique
>   package instead everything works fine. My question is more general here
>   though: *how to deal (access via reflection for example) with classes on
>   classpath that use same packages as certain modules on module path?* We
>   can't (it will be very difficult and inconvenient) ensure that no library
>   on classpath uses same package as some of the modules on module path...

Christian Stein (sormuras on Github) has already played with running JUnit5 on 
Java 9,
see https://github.com/sormuras/application-junit5-jdk9-demo

and for your second question, you can't, a package should be in only one module 
at a time (a package in the classpath is part of the unamed module), that's why 
you have to use --path-module to put all the test classes with the tested 
classes in the same package inside the same module.

> 
> Could you please clarify these aspects to me please or point me where I can
> read about it?
> 
> Thank you,
> Oleg

cheers,
Rémi


Re: javap cannot read module-info.class

2017-07-01 Thread Remi Forax
Hi Stephan,
there is something wrong from your side, it works for me :)

/usr/jdk/jdk-9-b175/bin/javap --module-path target/main/exploded/ --module 
fr.umlv.asm.test module-info
Compiled from "module-info.java"
module fr.umlv.asm.test {
  requires java.base;
  requires org.objectweb.asm.all.debug;
  exports fr.umlv.asm.test;
}

/usr/jdk/jdk-9-b175/bin/javap -m java.base module-info
Compiled from "module-info.java"
module java.base@9 {
  exports sun.reflect.generics.reflectiveObjects to
java.desktop;
  exports java.util;
  exports java.lang.ref;
  exports jdk.internal.misc to
java.rmi,
java.sql,
jdk.jshell,
...

cheers,
Rémi

- Mail original -
> De: "Stephan Herrmann" 
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Samedi 1 Juillet 2017 17:08:20
> Objet: javap cannot read module-info.class

> compile an arbitrary module-info.java and then:
> 
> $ javap module-info.class
>  Error: error while reading constant pool for /tmp/bin/module-info.class:
>  unexpected tag at #5: 19
> 
> From tweaking the Eclipse compiler it seems that javap is expecting a
> CONSTANT_Utf8_info,
> where according to JVMS 4.7.25 a CONSTANT_Module_info is required.
> 
> JVMS and javac agree, so javap is the odd man out.
> 
> Stephan
> 
> PS: build is 9+175


Re: Non Standadrd Module Attribute

2017-06-28 Thread forax
- Mail original -
> De: "Alan Bateman" 
> À: "Remi Forax" , "jigsaw-dev" 
> 
> Envoyé: Mercredi 28 Juin 2017 14:22:23
> Objet: Re: Non Standadrd Module Attribute

> On 28/06/2017 12:12, Remi Forax wrote:
>> Hi Alan, hi all,
>> i've implemented the non standard attributes ModuleTarget, ModuleHashes and
>> ModuleResolution in ASM.
>>
>> First, given that i was not able to find a document that describe them, i've
>> used the javadoc of the classes inside 
>> jdk.internal.module.ClassFileAttributes.
> JEP 261 needs a big update and I think is the right place to document
> the JDK-specific class file attributes (except maybe ModuleResolution
> where they could be a case to document it in JEP 11 instead).
> 
>> I've discovered that the jdk modules have an attribute ModuleTarget that can
>> have a null platform, why not removing the ModuleTarget attribute from the
>> module-info instead ?
> The ModuleTarget attribute is removed at link time from all modules
> except java.base (need to retain it in java.base to allow it be compared
> with any platform specific modules on the module path). The jlink system
> modules plugin has an option to retain the ModuleTarget attribute if you
> want.
> 
> As regards allowing the target_platform to be 0 (meaning not present)
> then it may be a left over from when the attribute had multiple values -
> it was possible for it to only have the OS name or architecture for example.

yes, it's what i'm believing too.
To take an example, currently, the module-info of java.sql as n attribute 
ModuleTarget which is empty, IMO only java.base should have a module attribute 
ModuleTarget.

> 
>> And for the module jdk.incubator.httpclient, the value of the resolution is 
>> 9,
>> so it means DO_NOT_RESOLVE_BY_DEFAULT | WARN_INCUBATING;
>> What is the exact meaning of DO_NOT_RESOLVE_BY_DEFAULT ?
>>
> From JEP 11 "incubator modules are not resolved by default for
> applications on the class path".

in that case why java.xml.bind which is not resolved by default too, does not 
have an attribute ModuleResolution ? 

> 
> -Alan

Rémi


Non Standadrd Module Attribute

2017-06-28 Thread Remi Forax
Hi Alan, hi all,
i've implemented the non standard attributes ModuleTarget, ModuleHashes and 
ModuleResolution in ASM.

First, given that i was not able to find a document that describe them, i've 
used the javadoc of the classes inside jdk.internal.module.ClassFileAttributes.
I've discovered that the jdk modules have an attribute ModuleTarget that can 
have a null platform, why not removing the ModuleTarget attribute from the 
module-info instead ?
And for the module jdk.incubator.httpclient, the value of the resolution is 9, 
so it means DO_NOT_RESOLVE_BY_DEFAULT | WARN_INCUBATING;
What is the exact meaning of DO_NOT_RESOLVE_BY_DEFAULT ?

regards,
Rémi



Re: 8182482: Module System spec updates

2017-06-21 Thread Remi Forax
- Mail original -
> De: "Alex Buckley" 
> À: "jigsaw-dev" 
> Envoyé: Mardi 20 Juin 2017 20:09:14
> Objet: Re: 8182482: Module System spec updates

> Hi Remi,
> 
> On 6/20/2017 6:29 AM, fo...@univ-mlv.fr wrote:
>> ok, let's focus on abstract class defining a service.
> 
> I would be happy for the "Designing services" section to give more
> advice about the tradeoffs between an interface and an abstract class.
> Two sentences, written in a style that leads a junior developer but does
> not judge them if they don't follow the advice. Can you write it? :-

sure let's try:
if you are wondering when to use an interface and when to use an abstract 
class, you can use this advice: most of the time you should favor interfaces, 
unless you want to execute your own code as part of any service implementations 
like by example when you want to add a security check to validate something 
about the implementation.

I'm sure you will be able to transform it in proper English.

> 
> -
> A service is a single type, usually an interface or abstract class.
> ***REMI'S TEXT HERE*** A concrete class can be used, but this is not
> recommended. The type may have any accessibility.
> 
> The methods of a service are highly ...
> -
> 
> Alex

Rémi


Re: 8182482: Module System spec updates

2017-06-20 Thread forax
- Mail original -
> De: "Alan Bateman" 
> À: "Remi Forax" 
> Cc: "jigsaw-dev" 
> Envoyé: Mardi 20 Juin 2017 13:30:20
> Objet: Re: 8182482: Module System spec updates

> On 20/06/2017 11:39, Remi Forax wrote:
>> Hi Alan,
>> the doc is clearly better.
>>
>> Can you change
>>A service is a single type, usually an interface or abstract class. A 
>> concrete
>>class can be used, but this is not recommended.
>> to
>>A service is a single type, usually an interface. An abstract class or a
>>concrete class can be used, but this is not recommended.
>>
>> An abstract class has a constructor so you can observe when a service
>> implementation is instantiated by the ServiceLoader, which may make the code
>> dependent of the loading order of the service implementations. So using an
>> abstract class for a service implementation is also a bad idea.
>>
> There shouldn't be any issue using an abstract class as the service.
> Sure, most developers will choose an interface but we have many examples
> where the service type is an abstract class (and there are corner cases
> with security where enforcement isn't possible when using an interface).

Abstract classes are evil when the abstract class and the concrete class are 
not maintained by the same people,
if the code of the abstract class change for whatever reason, it will break the 
subclasses. It's just bad design.

And the code that enforce security do not have to be in the abstract class 
constructor, debugging a constructor in general is harder than any other 
methods because the object is partially initialized. When you are executing the 
constructor of an abstract class the object corresponding to this is not fully 
initialized. You can always 'layer up' and use factories instead.
 
> 
> The service provider can be abstract class too but only when it defines
> a public static provider method.

yes, here, it's less problematic because in that case the abstract class and 
all subclasses will be maintained together.

> 
> I don't understand your point about an abstract class having a
> constructor. If it doesn't a static provider method then it won't
> compile as a module (it won't run either as it can't be instantiated).

ok, let's focus on abstract class defining a service.

> 
> -Alan

Rémi


Re: 8182482: Module System spec updates

2017-06-20 Thread Remi Forax
Hi Alan,
the doc is clearly better.

Can you change
  A service is a single type, usually an interface or abstract class. A 
concrete class can be used, but this is not recommended.
to 
  A service is a single type, usually an interface. An abstract class or a 
concrete class can be used, but this is not recommended.

An abstract class has a constructor so you can observe when a service 
implementation is instantiated by the ServiceLoader, which may make the code 
dependent of the loading order of the service implementations. So using an 
abstract class for a service implementation is also a bad idea.

regards,
Rémi   

- Mail original -
> De: "Alan Bateman" 
> À: "jigsaw-dev" 
> Envoyé: Mardi 20 Juin 2017 12:20:16
> Objet: 8182482: Module System spec updates

> We have two javadoc/spec updates that I'd like to get into the JDK 9
> Initial Release Candidate that is scheduled for this week.
> 
> The spec updates are for two issues:
> 
> 1. ServiceLoader: The API spec has been updated significantly to support
> modules but it needs another round of update to do clean-up to get it
> more readable and consistent, and also to align it with the JLS.  Most
> of reorganization and re-wording has been proposed by Alex. Joe Darcy
> has also proposed a few adjustments.
> 
> 2. Upgradable modules aren't specified anywhere. Java SE will designate
> a number of standard modules as upgradeable but we don't have anywhere
> in the docs to link to that or describe how the upgraded versions are
> used in preference to the modules built into the environment.
> 
> The webrev with the proposed (docs only, no implementation) changes is here:
>   http://cr.openjdk.java.net/~alanb/8182482/webrev/index.html
> 
> The ServiceLoader diffs are hard to read. It might be easier to read the
> generated javadoc:
> http://cr.openjdk.java.net/~alanb/8182482/docs/java/util/ServiceLoader.html
> 
> -Alan


Re: Review Request JDK-8182032: Make java.compiler upgradeable

2017-06-13 Thread Remi Forax
My concern is that by example javac and javap need to be aligned and by making 
a module upgradeable you can not make only some packages upgradeable.
So it seems logical that if java.compiler is upgradeable the other modules of 
langtools are upgradable too.

Rémi 


On June 13, 2017 10:26:18 AM GMT+02:00, Alan Bateman  
wrote:
>On 13/06/2017 08:42, Remi Forax wrote:
>> Hi Mandy,
>> why only java.compiler is upgradable and not all modules defined in
>langtools ?
>>
>java.compiler needs to be upgradeable because JSR 199 and JSR 269 are 
>standalone technologies.
>
>The other modules in the langtools repository are JDK-specific. They 
>aren't meant to be upgraded via the upgrade module path. If someone is 
>forking the code to create their own version of jdk.compiler (for 
>example) then they can re-package or load it into a child layer.
>
>-Alan

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.


Re: Review Request JDK-8182032: Make java.compiler upgradeable

2017-06-13 Thread Remi Forax
Hi Mandy,
why only java.compiler is upgradable and not all modules defined in langtools ?

cheers,
Rémi

- Mail original -
> De: "Mandy Chung" 
> À: "jigsaw-dev" 
> Envoyé: Mardi 13 Juin 2017 07:35:04
> Objet: Review Request JDK-8182032: Make java.compiler upgradeable

> Webrev at:
>   http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8182032/webrev.00/
> 
> java.compiler is a standalone technology that allows to be running on older 
> JDK,
> in particular for IDE to support new language features.
> 
> This patch takes out the make logic to find the modules that directly and
> indirectly require any upgradeable modules and include them as upgradeable.
> Instead it lists all upgradeable modules.  The list of upgradeable modules is
> small whereas the exclude list is not (which is also error-prone to find 
> them).
> I have added a new test to verify what modules are hashed in java.base.
> 
> Mandy


Re: Annotation-based syntax for module-info.java

2017-06-12 Thread Remi Forax
Hi Markus,

- Mail original -
> De: "Markus Keller" 
> À: jigsaw-dev@openjdk.java.net
> Cc: "Daniel Megert" 
> Envoyé: Lundi 12 Juin 2017 15:26:10
> Objet: Annotation-based syntax for module-info.java

> [I know this is coming way too late, and my recent accident that
> incapacitated me for several weeks didn't help either. Nevertheless, I
> think the current module-info.java syntax was a snap decision that should
> not end up in the Java 9 spec.]

i hope you're well now,
and as you said, it's too late, and it doesn't solve the issue we had (see 
below ...)

> 
> The discussions about restricted keywords
> http://openjdk.java.net/projects/jigsaw/spec/issues/#RestrictedKeywords
> and the difficulties with defining a proper grammar for module-info.java
> indicate that something could be fundamentally wrong with the chosen
> approach that tries to come up with a new way of specifying a language.

The problem is not to define the grammar, the module-info grammar is very 
straight forward.
The problem is how to introduce new keywords when the language evolves without 
breaking the source compatibility as we have done with assert and enum in the 
past.

Even if we can use annotations (that's ugly but it works) to solve the 
particular case of the module-info, we will have the same issue when we will 
want to introduce new keywords like 'var', 'data', 'closed', 'value'*, etc in 
the future.
Local keyword (named restricted keyword in the spec) is a good answer for 
specifying how a keyword and an identifier can have the same name and 
disambiguate them.

cheers,
Rémi

* those keywords are among the ones proposed by projects Amber and Valhalla for 
the next releases of Java.


RE: Will split-packages ever be supported?

2017-05-30 Thread Remi Forax


On May 30, 2017 4:28:00 PM GMT+02:00, Stephen Felts  
wrote:

Hi Stephen,

>Wouldn't it be possible to add an enhancement to allow for a module to
>add a package to an existing module? 

You can already use --patch-module at compile time and runtime and obviously 
inject packages before creating the module DAG if you create a ModuleLayer.

> Sort of like OSGI fragment
>bundles, which are very popular.

popular among OSGI users, and  sometimes for the wrong reason IMO like a 
practical workaround of the one artifact == one classloader limitation of OSGI

>I don't understand what is so fundamental about this.

OSGI requires packages, not modules, so fragments make a lot of more sense.

In JPMS, the identity of a module is its containing packages (even the non 
exported ones at least for 9) so fragment makes even less sense because they 
will change the ABI of a module.

Also note that the configuration adaptability use cases of fragments is covered 
by different mechanisms, services (using an interface instead of injecting 
property files is cleaner IMO), the require static (to react if a class is 
present or not by reflection), the multi version jars (for features depending 
on the JDK version) and jmod file (for including native libraries).

cheers,
Rémi 

>
>
>-Original Message-
>From: Alan Bateman 
>Sent: Tuesday, May 30, 2017 8:17 AM
>To: wzberger; jigsaw-dev@openjdk.java.net
>Subject: Re: Will split-packages ever be supported?
>
>On 30/05/2017 11:52, wzberger wrote:
>
>> Our project has around 45 jar libraries and we currently using 
>> split-packages as simplest modularization concept. It's a desktop 
>> project with different kind of controls - all have in common that
>they 
>> are located in the same package e.g. 'com.swing'. So users can add 
>> only required libraries to their project. However, in Jigsaw the 
>> split-package concept is not supported - so we have to completely 
>> rework our package structure. This means:
>>
>> - the new package structure will become more complicated because we 
>> have to add new packages
>> - our API isn't backward compatible
>> - our users have to rework their applications
>> - our users have to learn the new API (package structure)
>>
>> So how likely is it that split packages will be supported in the near
>
>> future?
>It's fundamental to reliable configuration that two or more modules do
>not export a package with the same name to a module that reads both
>(continuing from the spec "This includes the case where a module M
>containing package p reads another module that exports p to M"). It
>seems very unlikely to me that something as fundamental and core as
>this will ever be dropped.
>
>To your example, then if the project can't be restructured to address
>the split packages issues then it will need to stay on the class path. 
>There's nothing wrong with that, it should continue to work as before.
>
>-Alan.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.


Re: Will split-packages ever be supported?

2017-05-30 Thread Remi Forax
- Mail original -
> De: "Jochen Theodorou" 
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Mardi 30 Mai 2017 15:16:46
> Objet: Re: Will split-packages ever be supported?

> On 30.05.2017 14:16, Alan Bateman wrote:
>> On 30/05/2017 11:52, wzberger wrote:
>>
>>> Our project has around 45 jar libraries and we currently using
>>> split-packages as simplest modularization concept. It's a desktop
>>> project with different kind of controls - all have in common that they
>>> are located in the same package e.g. 'com.swing'. So users can add
>>> only required libraries to their project
> [...]
>> To your example, then if the project can't be restructured to address
>> the split packages issues then it will need to stay on the class path.
>> There's nothing wrong with that, it should continue to work as before.
> 
> Of course staying on the classpath means to get replaced in the future.
> So I don't agree with you here: there is a lot wrong with that imho.
> Nobody really cares in the end if you are used as module or not, but if
> you cannot do it, you are being replaced with something that can,
> because you are not future proof. Breaking the code of about everyone
> depending of you can have of course the same effect.
> 
> I mean... assume you want to write a named module... how do you depend
> on a class in an unnamed module? You do not. You can for example not
> extend such a class. javac would not allow that. I can of course still
> compile in classpath mode and then use a yet to be written tool, that
> generates the module information independent of javac... It means your
> module would have an undeclared requirement. I think that is a
> complication you normally would not want to have. It is probably more
> easy to drop the framework and use something else then.
> 
> Oh, you forgot the other alternative... make it a big monolith and
> squash the 45 jars into one big module. Then it can move away from the
> classpath too. Sure, making a monolithic module really defies the
> purpose of the module system, but it is still better than breaking code
> or requiring the users to do module system acrobatics.

the big monolithic module is a good transition solution,
i've done that with Aether to get access to Maven Central inside a fully 
modularized project,
but it's like with automatic modules, it's just a temporary solution that you 
can use in your application and not something you can do if you want to publish 
a library.

> 
> bye Jochen


Rémi


Re: JPMS Access Checks, Verification and the Security Manager

2017-05-23 Thread Remi Forax
Hi Volker,
this is the behavior of the verifier since 6 (when the split verifier becomes 
the default one).

Let say you have a code like this,

public class TestVerifier {
  static class B extends A { }
  static class A { }

  public static void main(String[] args) {
A a;
if (args.length == 0) {
  a = new A();
} else {
  a = new B();
}
System.out.println(a.toString());
  }
}

if you compile and then remove TestVerifier$B.class, you will get
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: TestVerifier$B
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at 
sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: TestVerifier$B
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more

whatever the comand line arguments because it's the verifier that throws an 
error. 

There is a workaround for you, if you use an interface instead of a class (in 
my example, make A an interface), it will work !
The verifier does not verify interface at verification time, the checks in done 
once by the VM at runtime.

so if there is a bug, it's in the way the verifier actually works i.e. if it 
can not find a class or do not see a class because of the module encapsulation, 
it should postpone the check at runtime instead of reporting an error at 
compile time. Thats said, i'm not sure it's a good idea.

cheers,
Rémi

- Mail original -
> De: "Volker Simonis" 
> À: "jigsaw-dev" , "security-dev" 
> 
> Envoyé: Mardi 23 Mai 2017 09:27:22
> Objet: JPMS Access Checks, Verification and the Security Manager

> Hi,
> 
> my question is if it is OK for a program which otherwise runs fine to
> fail during class verification if running with a security manager
> because of module access restrictions?
> 
> As the following write-up got a little lengthy, I've also uploaded a
> html version for a nicer reading experience :)
> 
> http://cr.openjdk.java.net/~simonis/webrevs/2017/veri_with_secman/sm.html
> 
> Consider the following small, self-contained program which statically
> references com.sun.crypto.provider.SunJCE:
> 
> import java.security.Provider;
> import com.sun.crypto.provider.SunJCE;
> 
> public class Tricky {
> 
>  public static void main(String args[]) throws Exception {
>try {
>  System.out.println(Tricky.class + " (" +
> Tricky.class.getClassLoader() + ")" + args[0]);
>}
>catch (Exception e) {
>  Provider p = new SunJCE();
>  System.out.println(p + " (" + p.toString() + ")");
>}
>  }
> }
> 
> This program easily compiles and runs with Oracle/OpenJDK 8:
> 
> $ javac Tricky
> $ java Tricky ""
> class Tricky (sun.misc.Launcher$AppClassLoader@4e0e2f2a)
> $ java Tricky
> SunJCE version 1.8 (SunJCE version 1.8)
> 
> The second invocation (without argument) will cause an exception (when
> trying to access the first element of the zero length argument array
> at 'args[0]') which will be caught in the exception handler where we
> create a new 'SunJCE' object and print its string representation to
> stdout. The first invocation (with an empty argument) doesn't provoke
> an exception and will just print the class itself together with its
> class loader.
> 
> When we run the same (jdk8 compiled) program with jdk9, we'll see the
> following result:
> 
> $ jdk9/java Tricky ""
> class Tricky (jdk.internal.loader.ClassLoaders$AppClassLoader@ba8a1dc)
> $ jdk9/java Tricky
> Exception in thread "main" java.lang.IllegalAccessError: class Tricky
> (in unnamed module @0x3b192d32) cannot access class
> com.sun.crypto.provider.SunJCE (in module java.base) because module
> java.base does not export com.sun.crypto.provider to unnamed module
> @0x3b192d32
>  at Tricky.main(Tricky.java:11)
> $ jdk9/java --add-exports java.base/com.sun.crypto.provider=ALL-UNNAMED Tricky
> SunJCE version 9 (SunJCE version 9)
> 
> The first invocation (with an empty argument) still works because we
> have lazy class loading and 'SunJCE' provider is actually never used.
> The second invocation (without argument) obviously fails with jdk9
> because 'java.base' doesn't export the package com.sun.crypto.provider
> anymore. This can be fixed by adding '--add-exports
> java.base/com.sun.crypto.provider=ALL-UNNAMED' to the command line as
> demonstrated in the third invocation.
> 
> So far so good -

Re: An alternative to "restricted keywords" + helping automatic modules

2017-05-20 Thread Remi Forax
Hi Stephen,

- Mail original -
> De: "Stephen Colebourne" 
> À: "jigsaw-dev" 
> Envoyé: Vendredi 19 Mai 2017 10:58:01
> Objet: Re: An alternative to "restricted keywords" + helping automatic modules

> I don't support the ^ element or escaping like that either.
> 
> However, would adding a "module" keyword help?
> 
> module com.foo.lib {
>  requires module com.foo.bar;
> }
> 
> thus:
> 
> module com.foo.lib {
>  requires static module blah;
>  requires transitive module transitive;
> }
> 
> ie. the module name is always prefixed by "module" in a "requires"
> statement. But does this help?

I will only answer on this question from the technical point of view, the EG as 
already rule out this syntax and i do not think it's wise to reopen an issue on 
the module-info syntax now.
So yes it helps but we don't care.

Adding any mandatory keyword between the modifiers and the module name in the 
require directive as the effect to allow to require a module with module name 
that starts with transitive.
But this is solve a corner case of a corner case.

The corner case we are trying to solve here is to allow package name and module 
name that contains any keywords defined in module-info that are not defined as 
keywords in JLS8, (open, module, requires, transitive, exports, opens, to, 
uses, provides, and with). For that we need, a way to activate keywords locally 
when the grammar wait for them and do not activate them where we don't care (in 
the middle of a package name or a module name).

The fact that activating keywords do not solve the problem of module that 
starts with transitive in the require directive should be seen in my point of 
view a corner case.
And as i already said, because module name should use reverse DNS convention, 
i'm not even sure this problem will occur in practice,
thus here we can just say that we will not solve that issue.

> 
> Stephen
> 
> 

regards,
Rémi


> On 18 May 2017 at 09:59, Stephan Herrmann  wrote:
>> Remi,
>>
>> I see your proposal as a minimal compromise, avoiding the worst
>> of difficulties, but I think we can do better.
>>
>> Trade-off:
>> In all posts I could not find a real reason against escaping,
>> aside from aesthetics. I don't see this as sufficient motivation
>> for a less-then-perfect solution.
>>
>>
>> Clarity:
>> I'm still not completely following your explanations, partly because
>> of the jargon you are using. I'll leave it to Alex to decide if he
>> likes the idea that JLS would have to explain terms like dotted
>> production.
>>
>> Compare this to just adding a few more rules to the grammar,
>> where no hand-waving is needed for an explanation.
>> No, I did not say that escaping is a pervasive change.
>> I never said that the grammar for ordinary compilation units
>> should be changed.
>> If you like we only need to extend one rule for the scope of
>> modular compilation units: Identifier. It can't get simpler.
>>
>>
>> Completeness:
>> I understand you as saying, module names cannot start with
>> "transitive". Mind you, that every modifier that will be added
>> to the grammar for modules in the future will cause conflicts for
>> names that are now legal, and you won't have a means to resolve this.
>>
>> By contrast, we can use the escaping approach even to solve one
>> more problem that has been briefly touched on this list before:
>>
>> Automatic modules suffer from the fact that some artifact names may
>> have Java keywords in their name, which means that these artifacts
>> simply cannot be used as automatic modules, right?
>> Why not apply escaping also here? *Any* dot-separated sequence
>> of words could be used as module name, as long as module references
>> have a means to escape any keywords in that sequence.
>>
>>
>> Suitability for implementation:
>> As said, your proposal resolves one problem, but still IDE
>> functionality suffers from restricted keywords, because scanning
>> and parsing need more context information than normal.
>> - Recovery after a syntax error will regress.
>> - Scanning arbitrary regions of code is not possible.
>> Remember:
>> In an IDE code with syntax errors is the norm, not an exception,
>> as the IDE provides functionality to work on incomplete code.
>>
>>
>> Stephan
>>
>>
>> On 18.05.2017 00:34, Remi Forax wrote:
>>>
>>> I want to answer this before we start the meetings because i really think
>>> that restricted keyword as i propose solve t

Re: An alternative to "restricted keywords" + helping automatic modules

2017-05-19 Thread forax
- Mail original -
> De: "Stephan Herrmann" 
> À: fo...@univ-mlv.fr, jigsaw-dev@openjdk.java.net
> Envoyé: Vendredi 19 Mai 2017 17:26:02
> Objet: Re: An alternative to "restricted keywords" + helping automatic modules

> Inline
> 
> On 19.05.2017 15:53, fo...@univ-mlv.fr wrote:
>>
>>
>> 
>>
>> *De: *"Stephan Herrmann" 
>> *À: *"John Rose" , jigsaw-dev@openjdk.java.net
>> *Cc: *"Rémi Forax" 
>> *Envoyé: *Vendredi 19 Mai 2017 12:37:07
>> *Objet: *Re: Re: An alternative to "restricted keywords" + helping 
>> automatic
>> modules
>>
>> A quick question to keep the ball rolling:
>>
>> Do we agree on the following assessment of the status quo?
>>
>>   The definition of "restricted keywords" implies (without explicitly 
>> saying so),
>>   that classification of a word as keyword vs. identifier can only be 
>> made
>>   *after* parsing has accepted the enclosing ModuleDeclaration.
>>   (With some tweaks, this can be narrowed down to
>>"after the enclosing ModuleDirective has been accepted")
>>
>>   This definition is not acceptable.
>>
>>
>> I agree that this is not acceptable but this is not what we are proposing.
> 
> Who is "we"?
> 
> Note that your proposal let me conclude that "transitive" is not a legal
> start of a module reference. If that is not what you intend, please provide
> a specification-like description of what you have in mind.
> Probably Stephen's proposal will come in handy for this issue?

transitive is not a valid start of a module name if you want to use it in a 
requires directive in Java,
but it's a valid module name for the JVM, you can create a module-info.class in 
another language than Java.

> 
> Your notes about possible implementation may help when we come to 
> implementing,
> but right now they may also distract from understanding the intention.

We have gone into the rabbit hole of talking about implementation because you 
ask to it's your point (3) ""restricted keywords" pose three problems to tool 
implementations".
The intention is to introduce restricted keywords (i prefer local keywords), to 
quote the the current draft of the JLS: "They are keywords solely where they 
appear as terminals in the ModuleDeclaration production (§7.7), and are 
identifiers everywhere else", so developers will not have to change all their 
Java codes because open, module, requires, transitive, exports, opens, to, 
uses, provides, and with are only keywords activated locally in 
module-info.java.

> 
> Stephan
> 
> 

Rémi

>>
>> You do not have to wait the reduction of ModuleDeclaration (or 
>> ModuleDirective),
>> the parser know its parsing state (the LR item)
>> during the parsing not at the end.
>> The LR analysis is not able to know at some point during the parsing which
>> production will be reduced later but it is able to know
>> which terminals will not lead to an error when shifting the next terminal.
>>
>> When you are in the middle of the parsing, the parser shift a terminal to go
>> from one state to another, so for a state the parser
>> knows if it can shift by a terminal which is among the set of restricted
>> keywords or not then either it can instruct the lexer
>> before scanning the token to activate the restricted keyword automata or 
>> after
>> having scanned the token it can classify the token as
>> a keyword instead of as an identifier.
>>
>> The idea is that the parser will not only tell when it reduces a production 
>> but
>> also when it is about to shift a restricted keyword.
>> So you can classify a token as an identifier or as a keyword because the 
>> parser
>> is able to bubble up that its parser state (the LR
>> item) may recognize a keyword.
>>
>>
>>
>> comments?
>> Stephan
>>
>>
>> Rémi
>>
>>
>> - ursprüngliche Nachricht -
>>
>> Subject: Re: An alternative to "restricted keywords" + helping automatic 
>> modules
>> Date: Fr 19 Mai 2017 07:27:31 CEST
>> From: John Rose
>> To: Stephan Herrmann
>>
>> On May 18, 2017, at 1:59 AM, Stephan Herrmann > <mailto:stephan.herrm...@berlin.de>> wrote:
>>
>>
>> In all posts I could not find a real reason

Re: An alternative to "restricted keywords" + helping automatic modules

2017-05-19 Thread forax
> De: "Stephan Herrmann" 
> À: "John Rose" , jigsaw-dev@openjdk.java.net
> Cc: "Rémi Forax" 
> Envoyé: Vendredi 19 Mai 2017 12:37:07
> Objet: Re: Re: An alternative to "restricted keywords" + helping automatic
> modules

> A quick question to keep the ball rolling:

> Do we agree on the following assessment of the status quo?

> The definition of "restricted keywords" implies (without explicitly saying 
> so),
> that classification of a word as keyword vs. identifier can only be made
> *after* parsing has accepted the enclosing ModuleDeclaration.
> (With some tweaks, this can be narrowed down to
> "after the enclosing ModuleDirective has been accepted")

> This definition is not acceptable.
I agree that this is not acceptable but this is not what we are proposing. 

You do not have to wait the reduction of ModuleDeclaration (or 
ModuleDirective), the parser know its parsing state (the LR item) during the 
parsing not at the end. 
The LR analysis is not able to know at some point during the parsing which 
production will be reduced later but it is able to know which terminals will 
not lead to an error when shifting the next terminal. 

When you are in the middle of the parsing, the parser shift a terminal to go 
from one state to another, so for a state the parser knows if it can shift by a 
terminal which is among the set of restricted keywords or not then either it 
can instruct the lexer before scanning the token to activate the restricted 
keyword automata or after having scanned the token it can classify the token as 
a keyword instead of as an identifier. 

The idea is that the parser will not only tell when it reduces a production but 
also when it is about to shift a restricted keyword. 
So you can classify a token as an identifier or as a keyword because the parser 
is able to bubble up that its parser state (the LR item) may recognize a 
keyword. 

> comments?
> Stephan

Rémi 

> - ursprüngliche Nachricht -

> Subject: Re: An alternative to "restricted keywords" + helping automatic 
> modules
> Date: Fr 19 Mai 2017 07:27:31 CEST
> From: John Rose
> To: Stephan Herrmann

> On May 18, 2017, at 1:59 AM, Stephan Herrmann < stephan.herrm...@berlin.de >
> wrote:

>> In all posts I could not find a real reason against escaping,
>> aside from aesthetics. I don't see this as sufficient motivation
>> for a less-then-perfect solution.

> So, by disregarding esthetics...

>> Clarity:
>> I'm still not completely following your explanations, partly because
>> of the jargon you are using. I'll leave it to Alex to decide if he
>> likes the idea that JLS would have to explain terms like dotted
>> production.

>> Compare this to just adding a few more rules to the grammar,
>> where no hand-waving is needed for an explanation.
>> No, I did not say that escaping is a pervasive change.
>> I never said that the grammar for ordinary compilation units
>> should be changed.
>> If you like we only need to extend one rule for the scope of
>> modular compilation units: Identifier. It can't get simpler.

>> Completeness:
>> I understand you as saying, module names cannot start with
>> "transitive". Mind you, that every modifier that will be added
>> to the grammar for modules in the future will cause conflicts for
>> names that are now legal, and you won't have a means to resolve this.

>> By contrast, we can use the escaping approach even to solve one
>> more problem that has been briefly touched on this list before:

>> Automatic modules suffer from the fact that some artifact names may
>> have Java keywords in their name, which means that these artifacts
>> simply cannot be used as automatic modules, right?
>> Why not apply escaping also here? *Any* dot-separated sequence
>> of words could be used as module name, as long as module references
>> have a means to escape any keywords in that sequence.

>> Suitability for implementation:
>> As said, your proposal resolves one problem, but still IDE
>> functionality suffers from restricted keywords, because scanning
>> and parsing need more context information than normal.

> …we obtain the freedom for IDEs to disregard abnormal
> amounts of context, saving uncounted machine cycles,

>> - Recovery after a syntax error will regress.

> …and we make life easier for all ten writers of error recovery
> functions,

>> - Scanning arbitrary regions of code is not possible.

> …we unleash the power of an army of grad students to study
> bidirectional parsing of module files,

>> Remember:
>> In an IDE code with syntax errors is the norm, not an exception,
>> as the IDE provides functionality to work on incomplete code.

> …and ease the burdens of the thousands who must spend their
> time looking at syntax errors for their broken module files.

> Nope, not for me. Give me esthetics, please. Really.

> — John

>  ursprüngliche Nachricht Ende 


Re: Attaching to a JVM image that does not include java.instrument

2017-05-19 Thread Remi Forax
- Mail original -
> De: "Alan Bateman" 
> À: "Rafael Winterhalter" 
> Cc: "jigsaw-dev" 
> Envoyé: Vendredi 19 Mai 2017 10:22:04
> Objet: Re: Attaching to a JVM image that does not include java.instrument

> On 19/05/2017 08:08, Rafael Winterhalter wrote:
> 
>> Hi Alan,
>>
>> I just retested with the most recent Java 9 snapshot and this is what
>> I get:
>>
>> rafael@rafc:~/jdk9-test/greetingsapp$ ./bin/java
>> -javaagent:/home/rafael/.m2/repository/sample-agent/sample-agent/1.0-SNAPSHOT/sample-agent-1.0-SNAPSHOT.jar
>> -m com.greetings/com.greetings.Main
>> Error occurred during initialization of VM
>> Could not find agent library instrument on the library path, with
>> error: libinstrument.so: cannot open shared object file: No such file
>> or directory
>> rafael@rafc:~/jdk9-test/greetingsapp$ ./bin/java --add-modules
>> java.instrument
>> -javaagent:/home/rafael/.m2/repository/sample-agent/sample-agent/1.0-SNAPSHOT/sample-agent-1.0-SNAPSHOT.jar
>> -m com.greetings/com.greetings.Main
>> Error occurred during initialization of VM
>> Could not find agent library instrument on the library path, with
>> error: libinstrument.so: cannot open shared object file: No such file
>> or directory
>> rafael@rafc:~/jdk9-test/greetingsapp$ ./bin/java --add-modules
>> java.instrument -m com.greetings/com.greetings.MainError occurred
>> during initialization of boot layer
>> java.lang.module.FindException: Module java.instrument not found
>>
>> Do I understand you correctly that you expect the last outcome also as
>> a result for the first command?
> If the run-time image does not contain java.instrument then the error
> for each of these cases should be like the last one ("Module
> java.instrument not found"). We have it right for -Dcom.sun.management.*
> and -XX:+EnableJVMCI but not -javaagent. The reason for the strange
> error with -javaagent is because java agents are implemented as a JVM TI
> agent that is loaded early in VM startup, long before the root modules
> are resolved. I will create a bug for this.
>>
>> I argue that this outcome would still be rather suprising for most
>> Java users. Many monitoring tools use Java agents to hook into Java
>> processes and those tools have become omnipresent in most production
>> environments. Also, it is very unlikely that a main application module
>> would require the java.instrument module as it is only useful to Java
>> agents. As a result, Java agents and the tooling that relies on them
>> would stop working for the majority of jlink modules. Furthermore, the
>> agent attachment also fails if a Javaagent does not require the
>> Instrumentation interface what makes this problem even less intuitive.
>> (The reason being the lack of libinstrument.)
> The jlink user is a power user and so decides the modules and features
> to include in the run-time image.
> 
> If the jlink user doesn't select the VM type when creating the run-time
> image then the attach mechanism will work (because it is compiled into
> libjvm.so) and the troubleshooting tools should work fine. However some
> features that are exposed via the attach mechanism may not be present,
> specifically VirtualMachine.startManagementAgent and
> VirtualMachine.startLocalManagementAgent will fail if module
> jdk.management.agent is not included, and VirtualMachine.loadAgent will
> fail if module java.instrument is not included.
> 
> If the jlink user is targeting an embedded system or wants to create a
> minimal image to put in a docker container then maybe they select the
> minimal VM, in which case the serviceability features (including the VM
> side of the attach mechanism) are not included. It gets harder again for
> troubleshooting when additional jlink power options are used to strip
> debug attributes. That's the trade-off.
> 
> As I said, the goal has always been to allow someone create a run-time
> image that only contains java.base. I'm not sure that subsuming
> java.instrument into java.base is right. Introducing options to ask
> jlink to exclude modules creates the potential for conflict on the
> command line. One thing that jlink could do is emit a warning that the
> resulting run-time image doesn't have the management and instrumentation
> features, might that be the right balance.

yes, that seems the right way to solve this issue.

> 
>>
>> On the above machine, a jlink image is incapable of starting despite
>> the _JAVA_OPTIONS only specifying standard features that are
>> guaranteed by the JVM independently of a specific VM vendor.
> There isn't any guarantee, a point that was clarified in the
> java.instrument spec in Java SE 8 (JDK-8006565). The motivation for the
> clarification at the time was embedded deployments and the Compact
> Profiles defined for Java SE 8 (compact1 and compact2 didn't include the
> java.lang.instrument package).
> 
> -Alan

Rémi


Re: An alternative to "restricted keywords" + helping automatic modules

2017-05-18 Thread forax
- Mail original -
> De: "Stephan Herrmann" 
> À: "Remi Forax" , jigsaw-dev@openjdk.java.net
> Envoyé: Jeudi 18 Mai 2017 10:59:09
> Objet: Re: An alternative to "restricted keywords" + helping automatic modules

> Remi,

Stephan,

>
> I see your proposal as a minimal compromise, avoiding the worst
> of difficulties, but I think we can do better.

better is usually a bitter enemy

>
> Trade-off:
> In all posts I could not find a real reason against escaping,
> aside from aesthetics. I don't see this as sufficient motivation
> for a less-then-perfect solution.
>
>
> Clarity:
> I'm still not completely following your explanations, partly because
> of the jargon you are using. I'll leave it to Alex to decide if he
> likes the idea that JLS would have to explain terms like dotted
> production.

Sorry for the jargon, dotted production is the same thing as a LR parser item 
[1],
the dot mark the parsing position inside a production.
i used 'dotted production' instead of parser item because usually it's clearer 
for my students.

>
> Compare this to just adding a few more rules to the grammar,
> where no hand-waving is needed for an explanation.
> No, I did not say that escaping is a pervasive change.
> I never said that the grammar for ordinary compilation units
> should be changed.
> If you like we only need to extend one rule for the scope of
> modular compilation units: Identifier. It can't get simpler.
>

I do not like ^ because
- as John said, esthetics is important
- it pushes the burden to the developers and not to the guys that implements 
the grammar.
  I'm Ok to makes your life and the life of people that implement the Java 
grammar (me included) less fun if for all other Java developers it just works, 
given the scale of the Java community, it seems to be a good compromise.
- ^ has to be a pervasive change, i mean it can be specified as a change only 
for module-info but from the developers point of view, it will be weird if you 
introduce ^ in module-info and not introduce it in the whole grammar
  so it's a global solution to local problem.  

so in my opinion, it's not that ^ does not work, as you said, it works in 
Xtend, it's that ^ is a escape hatch, it's better to use it when all other 
solutions do not work.

>
> Completeness:
> I understand you as saying, module names cannot start with
> "transitive". Mind you, that every modifier that will be added
> to the grammar for modules in the future will cause conflicts for
> names that are now legal, and you won't have a means to resolve this.
>
> By contrast, we can use the escaping approach even to solve one
> more problem that has been briefly touched on this list before:
>
> Automatic modules suffer from the fact that some artifact names may
> have Java keywords in their name, which means that these artifacts
> simply cannot be used as automatic modules, right?
> Why not apply escaping also here? *Any* dot-separated sequence
> of words could be used as module name, as long as module references
> have a means to escape any keywords in that sequence.
>
>
> Suitability for implementation:
> As said, your proposal resolves one problem, but still IDE
> functionality suffers from restricted keywords, because scanning
> and parsing need more context information than normal.
> - Recovery after a syntax error will regress.

Error recovery will not regress in all existing java file because restricted 
keyword only works when parsing the module-info.
And technically, there is no regression possible because the module-info was 
not existing before.
So error recovery after a syntax error in a module-info may be less fun to 
handle, as i said above, i'm ok with that.


> - Scanning arbitrary regions of code is not possible.

Scanning an arbitrary region is not easy in general, by example, you have if 
you are inside or outside a string, so you have to keep some information to be 
able to scan a region, why not trying to keep the parser state when necessary.
As John said, it seems to be a nice problem for grad students and at worst, you 
can use the existing code, it will display a restricted keyword in bold in the 
middle of a package name, that's all. 

> Remember:
> In an IDE code with syntax errors is the norm, not an exception,
> as the IDE provides functionality to work on incomplete code.
>
>
> Stephan

Rémi

[1] https://en.wikipedia.org/wiki/Canonical_LR_parser

>
>
> On 18.05.2017 00:34, Remi Forax wrote:
>> I want to answer this before we start the meetings because i really think 
>> that
>> restricted keyword as i propose solve the issues Stephan raised.
>>
>>
>> - Mail original -
>>> De: &

Re: An alternative to "restricted keywords"

2017-05-17 Thread Remi Forax
I want to answer this before we start the meetings because i really think that 
restricted keyword as i propose solve the issues Stephan raised.


- Mail original -
> De: "Stephan Herrmann" 
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Mardi 16 Mai 2017 11:49:45
> Objet: Re: An alternative to "restricted keywords"

> Thanks, Remi, for taking this to the EG list.
> 
> Some collected responses:
> 
> 
> Remi: "from the user point of view, '^' looks like a hack"
> 
> This is, of course, a subjective statement. I don't share this view
> and in years of experience with Xtext-languages (where this concept
> is used by default) I never heard any user complain about this.
> 
> More importantly, I hold that such aesthetic considerations are of
> much lesser significance than the question, whether we can explain
> - unambiguously explain - the concept in a few simple sentences.
> Explaining must be possible at two levels: in a rigorous specification
> and in simple words for users of the language.

I'm not against ^, or ` as it has already asked to escape an identifier, but as 
you said it's a pervasive change that applies on the whole grammar while i 
think that with restricted keyword (that really should be called local 
keywords) the changes only impact the grammar that specifies a module-info.java

> 
> Remi: "a keyword which is activated if you are at a position in the
>  grammar where it can be recognized".
> 
> I don't think 'being at a position in the grammar' is a good way of
> explaining. Parsing doesn't generally have one position in a grammar,
> multiple productions can be active in the same parser state.
> Also speaking of a "loop" for modifiers seems to complicate matters
> more than necessary.
> 
> Under these considerations I still see '^' as the clearest of all
> solutions. Clear as a specification, simple to explain to users.

Eclipse uses a LR parser, for a LR parser, position == dotted production as i 
have written earlier, so no problem because it corresponds to only one parser 
state.  Note that even if one do not use an LR or a LL parser, most hand 
written parser i've seen, javac is one of them, also refers to dotted 
production in the comments of the corresponding methods.

> 
> 
> 
> Peter spoke about module names vs. package names.
> 
> I think we agree, that module names cannot use "module words",
> whereas package names should be expected to contain them.

yes, that the main issue, package names may contains unqualified name like 
'transitive, ''with' or 'to'.
but i think people will also want to use existing package or more exactly 
prefix of existing package as module name, so we should also support having 
restricted keyword name as part of a module name.

The grammar is:

  open? module module_name {
requires (transitive | static)* module_name;
exports package_name;
exports package_name to module_name1, module_name2;
opens package_name;
opens package_name to module_name1, module_name2;
uses xxx;
provides xxx with xxx, yyy;
  }

If we just consider package name, only 'opens' and 'exports' are followed by a 
package name and a package name can only been followed by ';' or 'to', so once 
'opens' is parsed, you know that you can have only an identifier so if it's not 
an identifier by one of the restricted keywords, it should be considered as an 
identifier.

As i said earlier, the scanner can see the restricted keyword as keyword and 
before feeding the token to the parser, you can check the parser state to see 
if the keyword as to be lowered to an identifier or not.

For module name, there is the supplementary problem of transitive, because if a 
module starts with transitive, you can have a conflict. As i said earlier, 
instead of using the next token to know if transitive is the keyword or part of 
the module name, i think we should consider it as a keyword, as the JLS said a 
restricted keyword is activated when it can appear, so "requires transitive" is 
not a valid directive.

> 
> Remi: "you should use reverse DNS naming for package so no problem :)"
> 
> "to" is a "module word" and a TLD.
> I think we should be very careful in judging that a existing conflict
> is not a real problem. Better to clearly and rigorously avoid the
> conflict in the first place.

to as the first part of a package/module and to as in exports ... to can not be 
present on the same dotted production, because exports as to be followed by a 
package_name so 'to' here means the start of a package name and then because a 
package name can not ends with '.' you always know if you are inside the 
production recognizing the package_name or outside matching the to of the 
directive exports. 

> 
> 
> 
> Some additional notes from my side:
> 
> In the escape-approach, it may be prudent to technically allow
> escaping even words that are identifiers in Java 9, but could become
> keywords in a future version. This ensures that modules which need
> more escaping in Java 9+X can stil

Re: An alternative to "restricted keywords"

2017-05-12 Thread Remi Forax
Hi Peter,

On May 12, 2017 6:08:58 PM GMT+02:00, Peter Levart  
wrote:
>Hi Remi,
>
>On 05/12/2017 08:17 AM, Remi Forax wrote:
>> [CC JPMS expert mailing list because, it's an important issue IMO]
>>
>> I've a counter proposition.
>>
>> I do not like your proposal because from the user point of view, '^'
>looks like a hack, it's not used anywhere else in the grammar.
>> I agree that restricted keywords are not properly specified in JLS.
>Reading your mail, i've discovered that what i was calling restricted
>keywords is not what javac implements :(
>> I agree that restricted keywords should be only enabled when parsing
>module-info.java
>> I agree that doing error recovery on the way the grammar for
>module-info is currently implemented in javac leads to less than ideal
>error messages.
>>
>> In my opinion, both
>> module m { requires transitive transitive; }
>> module m { requires transitive; }
>> should be rejected because what javac implements something more close
>to the javascript ASI rules than restricted keywords as currently
>specified by Alex.
>>
>> For me, a restricted keyword is a keyword which is activated if you
>are at a position in the grammar where it can be recognized and because
>it's a keyword, it tooks over an identifier.
>> by example for
>>module m {
>> if the next token is 'requires', it should be recognized as a keyword
>because you can parse a directive 'required ...' so there is a
>production that will starts with the 'required' keyword.
>>
>> so
>>module m { requires transitive; }
>> should be rejected because transitive should be recognized as a
>keyword after requires and the compiler should report a missing module
>name.
>>   
>> and
>>module m { requires transitive transitive; }
>> should be rejected because the grammar that parse the modifiers is
>defined as "a loop" so from the grammar point of view it's like
>>module m { requires Modifier Modifier; }
>> so the the front end of the compiler should report a missing module
>name and a later phase should report that there is twice the same
>modifier 'transitive'.
>>
>> I believe that with this definition of 'restricted keyword', compiler
>can recover error more easily and offers meaningful error message and
>the module-info part of the grammar is LR(1).
>
>This will make "requires", "uses", "provides", "with", "to", "static", 
>"transitive", "exports", etc  all illegal module names. Ok, no big 
>deal, because there are no module names yet (apart from JDK modules and
>
>those are named differently). But...

you should use reverse DNS naming for module name, so no problem. 

>
>What about:
>
>module m { exports transitive; }
>
>Here 'transitive' is an existing package name for example. Who 
>guarantees that there are no packages out there with names matching 
>restricted keywords? Current restriction for modules is that they can 
>not have an unnamed package. Do we want to restrict package names a 
>module can export too?

you should use reverse DNS naming for package so no problem :)

>
>Stephan's solution does not have this problem.
>
>Regards, Peter

I think those issues are not real problem. 

Rémi 

>
>>
>> regards,
>> Rémi
>>
>> - Mail original -
>>> De: "Stephan Herrmann" 
>>> À: jigsaw-dev@openjdk.java.net
>>> Envoyé: Mardi 9 Mai 2017 16:56:11
>>> Objet: An alternative to "restricted keywords"
>>> (1) I understand the need for avoiding that new module-related
>>> keywords conflict with existing code, where these words may be used
>>> as identifiers. Moreover, it must be possible for a module
>declaration
>>> to refer to packages or types thusly named.
>>>
>>> However,
>>>
>>> (2) The currently proposed "restricted keywords" are not
>appropriately
>>> specified in JLS.
>>>
>>> (3) The currently proposed "restricted keywords" pose difficulties
>to
>>> the implementation of all tools that need to parse a module
>declaration.
>>>
>>> (4) A simple alternative to "restricted keywords" exists, which has
>not
>>> received the attention it deserves.
>>>
>>> Details:
>>>
>>> (2) The current specification implicitly violates the assumption
>that
>>> parsing can be performed on the basis of a tok

Re: Why not multiple modules of the same name in a class loader?

2017-05-12 Thread Remi Forax


On May 12, 2017 8:08:38 PM GMT+02:00, "David M. Lloyd"  
wrote:
>This has come up a couple times now and I'm not sure why the rule 
>exists: why not allow multiple modules with the same name in the same 
>class loader?

module names appear in the stack traces,  it will make life of people miserable 
if they have to open several artifacts to find the good one.

and if later the VM is able to isolate un-exported packages, it will be a real 
mess because at worst, you will have to follow all stack elements to find the 
class you want.


>
>As far as I can tell, there is no way to locate a module by class 
>loader, and module namespaces are generally already well-governed by 
>their layer, so from an API perspective at least, it seems like this is
>
>a harmless scenario (as long as there are no package conflicts, 
>obviously, but a container generally is able to manage this concern).
>
>Preventing this does block one potential workaround for the lack of a 
>ModuleLayer.Controller.addPackage() method, which would be to
>supplement 
>a module's content at run time by defining a new Layer that contains a 
>module of the same name as the original content within the same class 
>loader, where the container would then take care of such details as 
>mutual read/opening and cyclic access.
>
>So that made me curious: is there some hotspot-internal dictionary that
>
>I missed, which manages the set of modules within a class loader?

take a look to defineModule :)

Remi

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.


Re: An alternative to "restricted keywords"

2017-05-11 Thread Remi Forax
[CC JPMS expert mailing list because, it's an important issue IMO]

I've a counter proposition.

I do not like your proposal because from the user point of view, '^' looks like 
a hack, it's not used anywhere else in the grammar. 
I agree that restricted keywords are not properly specified in JLS. Reading 
your mail, i've discovered that what i was calling restricted keywords is not 
what javac implements :(
I agree that restricted keywords should be only enabled when parsing 
module-info.java
I agree that doing error recovery on the way the grammar for module-info is 
currently implemented in javac leads to less than ideal error messages.

In my opinion, both
   module m { requires transitive transitive; }
   module m { requires transitive; }
should be rejected because what javac implements something more close to the 
javascript ASI rules than restricted keywords as currently specified by Alex.

For me, a restricted keyword is a keyword which is activated if you are at a 
position in the grammar where it can be recognized and because it's a keyword, 
it tooks over an identifier.
by example for 
  module m { 
if the next token is 'requires', it should be recognized as a keyword because 
you can parse a directive 'required ...' so there is a production that will 
starts with the 'required' keyword.

so 
  module m { requires transitive; }
should be rejected because transitive should be recognized as a keyword after 
requires and the compiler should report a missing module name.
 
and
  module m { requires transitive transitive; }
should be rejected because the grammar that parse the modifiers is defined as 
"a loop" so from the grammar point of view it's like
  module m { requires Modifier Modifier; }
so the the front end of the compiler should report a missing module name and a 
later phase should report that there is twice the same modifier 'transitive'.

I believe that with this definition of 'restricted keyword', compiler can 
recover error more easily and offers meaningful error message and the 
module-info part of the grammar is LR(1).

regards,
Rémi

- Mail original -
> De: "Stephan Herrmann" 
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Mardi 9 Mai 2017 16:56:11
> Objet: An alternative to "restricted keywords"

> (1) I understand the need for avoiding that new module-related
> keywords conflict with existing code, where these words may be used
> as identifiers. Moreover, it must be possible for a module declaration
> to refer to packages or types thusly named.
> 
> However,
> 
> (2) The currently proposed "restricted keywords" are not appropriately
> specified in JLS.
> 
> (3) The currently proposed "restricted keywords" pose difficulties to
> the implementation of all tools that need to parse a module declaration.
> 
> (4) A simple alternative to "restricted keywords" exists, which has not
> received the attention it deserves.
> 
> Details:
> 
> (2) The current specification implicitly violates the assumption that
> parsing can be performed on the basis of a token stream produced by
> a scanner (aka lexer). From discussion on this list we learned that
> the following examples are intended to be syntactically legal:
>module m { requires transitive transitive; }
>module m { requires transitive; }
> (Please for the moment disregard heuristic solutions, while we are
>  investigating whether generally "restricted keywords" is a well-defined
>  concept, or not.)
> Of the three occurrences of "transitive", #1 is a keyword, the others
> are identifiers. At the point when the parser has consumed "requires"
> and now asks about classification of the word "transitive", the scanner
> cannot possible answer this classification. It can only answer for sure,
> after the *parser* has accepted the full declaration. Put differently,
> the parser must consume more tokens than have been classified by the
> Scanner. Put differently, to faithfully parse arbitrary grammars using
> a concept of "restricted keywords", scanners must provide speculative
> answers, which may later need to be revised by backtracking or similar
> exhaustive exploration of the space of possible interpretations.
> 
> The specification is totally silent about this fundamental change.
> 
> 
> (3) "restricted keywords" pose three problems to tool implementations:
> 
> (3.a) Any known practical approach to implement a parser with
> "restricted keywords" requires to leverage heuristics, which are based
> on the exact set of rules defined in the grammar. Such heuristics
> reduce the look-ahead that needs to be performed by the scanner,
> in order to avoid the full exhaustive exploration mentioned above.
> A set of such heuristic is extremely fragile and can easily break when
> later more rules are added to the grammar. This means small future
> language changes can easily break any chosen strategy.
> 
> (3.b) If parsing works for error-free input, this doesn't imply that
> a parser will be able to give any useful answer for input with sy

Re: Need help implementing Java modules

2017-05-09 Thread Remi Forax


On May 10, 2017 2:20:31 AM GMT+02:00, Ralph Goers  wrote:
>
>> On May 9, 2017, at 3:39 PM, Alex Buckley 
>wrote:
>> 
>> On 5/9/2017 3:04 PM, Ralph Goers wrote:
>>> Pardon me for being dense, but my reading said that Java modules
>>> disallowed runtime cycles as well as compile time. Once LoggerFinder
>>> binds with the module that provides that service does that not
>create
>>> a runtime dependency?  I don’t recall seeing anything describing
>what
>>> the behavior of that is.
>> 
>> The module system disallows cycles in the 'requires' directives of
>module declarations. The module system allows cycles in the "reads"
>relation of run-time modules.
>> 
>> When java.base 'uses LoggerFinder;' and invokes ServiceLoader to find
>providers, there is indeed a "reads" cycle created between the provider
>module and java.base. ServiceLoader is not special in creating this
>cycle -- you can create them yourself with the two addReads methods in
>the API, and all automatic modules have cyclic readability. But there
>is no 'requires' cycle.
>> 
>> Alex
>> 
>
>Thanks Alex, that would indeed be enough to break the cycle as
>log4j-api can easily use serviceLoader to bind with log4j-core as that
>is a very minor change. However your previous response leads me to
>another question.
>
>Log4j already has a robust plugin approach for users to implement their
>own Appenders, Filters, Layouts, and other Log4j components. We are not
>going to modify that as it would severely impact our users who have
>already implemented custom components and what we have works very well.
>Although the FlumeAppender I mentioned previously is provided by Log4j
>it, and all other Log4j components, are located via an annotation
>processor provided by Log4j. The processor runs at compile time and
>generates a file named
>META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat.
>All of these files are then located at runtime using
>ClassLoader.getResources(). As we parse the configuration provided by
>the user we instantiate the plugins using the class names provided in
>those files and the static builder or static factory method provided in
>the plugin class. From what I am reading in the javadoc we should not
>have any trouble with this because META-INF is not a valid package name
>so the resource file will not be hidden. Log4j uses reflection to call
>the static builder or static method to create the plugin.  
>
>With all this in mind, if users create modules will they be required to
>declare the packages where they have created plugins as “open” to log4j
>for this to work?

It depends if when you call the static method you need to bypass the 
encapsulation or not i.e. if your current code uses setAccessible, yes, the 
plugin's module has to be opened.

In any case, you need to add a read edge at runtime from log4j to the plugin, 
otherwise you will not find the plugins class.

> I am assuming that Log4j will be able to access this
>module without having to declare a dependency on it? 

yes, if you add a read edge at runtime.

>
>Thanks,
>Ralph

cheers,
Rémi 

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.


--ignore-signing-information do not suppress all errors

2017-05-07 Thread Remi Forax
Hi all,
I wonder if it's an issue or if there is a rational for not suppressing this 
error when using --ignore-signing-information.

I'm patching the compiler related modules of the jdk9 with the one from 
amber/langtools, so i know that i'm doing something borderline but i gently ask 
jlink to do not bother me with the hash signature.

...
jlink --module-path ./target/main/artifact:./deps:/usr/jdk/jdk-9-b167/jmods
  --add-modules 
jdk.policytool,jdk.charsets,jdk.pack,jdk.xml.dom,java.xml,jdk.deploy,java.datatransfer,java.jnlp,java.desktop,com.github.forax.pro.uberbooter,java.se,jdk.snmp,java.security.sasl,jdk.deploy.controlpanel,jdk.zipfs,java.base,jdk.crypto.ec,com.github.forax.pro.main,jdk.management.agent,jdk.jshell,jdk.editpad,oracle.desktop,jdk.jsobject,jdk.unsupported,java.smartcardio,jdk.scripting.nashorn,java.security.jgss,com.github.forax.pro.plugin.convention,jdk.dynalink,java.activation,java.sql,java.logging,jdk.jfr,jdk.internal.vm.ci,com.github.forax.pro.plugin.packager,jdk.packager.services,jdk.net,javafx.controls,jdk.internal.ed,jdk.compiler,jdk.naming.rmi,jdk.jconsole,jdk.internal.le,jdk.packager,jdk.jdwp.agent,jdk.internal.vm.compiler,com.github.forax.pro.daemon.imp,java.scripting,jdk.jartool,java.rmi,jdk.jdi,jdk.javaws,jdk.rmic,com.github.forax.pro.plugin.resolver,jdk.jstatd,jdk.httpserver,java.se.ee,jdk.jcmd,javafx.base,com.github.forax.pro.plugin.linker,com.github.forax.pro.plugin.compiler,jdk.plugin.dom,com.github.forax.pro.plugin.modulefixer,jdk.javadoc,jdk.xml.ws,java.sql.rowset,jdk.sctp,javafx.swing,jdk.jlink,jdk.scripting.nashorn.shell,oracle.net,java.xml.bind,java.compiler,javafx.graphics,javafx.fxml,jdk.plugin,javafx.media,jdk.accessibility,jdk.security.jgss,javafx.web,com.github.forax.pro.plugin.uberpackager,jdk.hotspot.agent,javafx.deploy,java.xml.crypto,jdk.incubator.httpclient,jdk.plugin.server,jdk.crypto.cryptoki,java.naming,java.prefs,jdk.internal.opt,jdk.attach,java.transaction,java.xml.ws,java.xml.ws.annotation,java.management,jdk.xml.bind,jdk.internal.jvmstat,java.instrument,jdk.management,jdk.security.auth,jdk.jdeps,jdk.aot,java.corba,java.management.rmi,jdk.naming.dns,jdk.localedata
  --launcher pro=com.github.forax.pro.main/com.github.forax.pro.main.Main
  --compress 0
  --ignore-signing-information
  --output ./target/image
Error: Hash of jdk.javadoc 
(5b3a628bce2aad807392ea8a3f2ce69a652e885e33db586694b47fcf6f5ebf52) differs to 
expected hash 
(6bdf7f26f72c9f102c5052f8ddf68a80a4b18f997a16e56805e3acc7347469b5) recorded in 
java.base

cheers,
Rémi


Re: Revised proposal for #AutomaticModuleNames

2017-05-05 Thread Remi Forax
Hi Robert,

- Mail original -
> De: "Robert Scholte" 
> À: "mark reinhold" 
> Cc: jigsaw-dev@openjdk.java.net
> Envoyé: Vendredi 5 Mai 2017 12:04:15
> Objet: Re: Revised proposal for #AutomaticModuleNames

> Hi Mark,
> 
> thanks for these adjustments. In general they look good, but it all
> depends on the details:
> 
> Define a JAR-file manifest attribute, `Automatic-Module-Name`, whose
> value is used as the name of the automatic module defined by that JAR
> file when it is placed on the module path, as previously suggested
> [2][3].  If a JAR file on the module path does not have such a
> manifest attribute then its automatic-module name is computed using
> the existing filename-based algorithm.
> 
> Just to be clear: does this make it an auto module? In that case we're
> still a bit stuck, because I still insist that jars published to any
> public artifact repository should never refer to an automodule. Published
> jars with this attribute should have received this name by their developer
> and this also implies that the jar is Jigsaw-ready, e.g. no more split
> packages. In the future its users should be able to switch to the fully
> modular jar with the same name without any problem. So I need an extra
> state, e.g. isAutomatic() + isNamed(), in which case I have to change my
> insisting line: "Never publish jars to any public artifact repository that
> refers to *unnamed* automatic modules".

yes, knowing if the name is auto-calculated or not may be interesting,
that's said, you can already get the manifest file from the ModuleReader and 
check if there is an entry named Automatic-Module-Name so i'm not sure 
surfacing isNamed() in the API is a good idea.

> 
> A user of a library can suggest a stable name to the library's
> maintainer, and easily submit a patch that implements it.  This will
> enable the uncoordinated modularization of libraries across the
> entire ecosystem.
> 
> I don't think this is realistic. Jars live in the local repository and
> will stay there even during compilation. Is a user patches such jars,
> it'll effect all his local Java projects. Only when Maven will do both
> compiling and generation the distributable/deployable one could isolate
> jars. 

[...]

I think there is a misunderstanding here, 
i read 'easily submit a patch' as 'easily submit a pull request' that will 
change the source so the next time the jar is published on central, it will 
have a module name in its manifest.


> 
> thanks,
> Robert

Rémi

> 
> On Thu, 04 May 2017 19:39:06 +0200,  wrote:
> 
>> Thanks to everyone, and especially Stephen Colebourne, Brian Fox, and
>> Robert Scholte, for the extensive feedback.
>>
>>   
>> http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-May/000687.html
>>
>> TL;DR: Keep automatic modules, bring back the module-name JAR-file
>> manifest attribute, and strongly recommend reverse-DNS module names.
>>
>> Comments?
>>
> > - Mark


Re: Revised proposal for #AutomaticModuleNames

2017-05-04 Thread Remi Forax


On May 4, 2017 11:21:38 PM GMT+02:00, Stephen Colebourne  
wrote:
>On 4 May 2017 at 18:38,   wrote:

...

>>   - To increase awareness of when automatic modules are used, and of
>the
>> consequences of their use, suggest that Java language compilers
>issue
>> two new types of warnings, and implement these warnings in the
>RI:
>
>Perhaps these should be reconsidered? If the module depends on an
>automatic module with a module name in the manifest, it can be
>considered reasonably stable. Whereas anything depending on a filename
>derived module name is unstable. That isn't being captured by the
>warnings.

an automatic module is still not stable because it exports all packages by 
default while a real module may only export some of them.

 
>
>Stephen

Rémi 

>
>
>On 4 May 2017 at 18:39,   wrote:
>> Thanks to everyone, and especially Stephen Colebourne, Brian Fox, and
>> Robert Scholte, for the extensive feedback.
>>
>>  
>http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-May/000687.html
>>
>> TL;DR: Keep automatic modules, bring back the module-name JAR-file
>> manifest attribute, and strongly recommend reverse-DNS module names.
>>
>> Comments?
>>
>> - Mark

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.


Re: Java Platform Module System

2017-05-03 Thread forax
- Mail original -
> De: "Stephan Herrmann" 
> À: jigsaw-dev@openjdk.java.net, "Remi Forax" , "Alex 
> Buckley" 
> Cc: "Brian Goetz" , "Dan Smith" 
> 
> Envoyé: Mercredi 3 Mai 2017 23:31:14
> Objet: Re: Java Platform Module System

> On 03.05.2017 20:55, Remi Forax wrote:
> > It's context-free because a context free grammar defined its input in term 
> > of
> > terminals and the theory do not say how to map a token to a terminal.
> >
> > Jay is right that it requires to use either some specific parser generator
> > like Tatoo [1] the one i've written 10 years ago (because i wanted the tool 
> > to
> > help me to extend a grammar easily) or to modify an existing parser 
> > generator so
> > the parser can send the production state to the lexer which will 
> > enable/disable
> > the automata that recognize the associated keywords .
> 
> Just feeding parser state into the Lexer doesn't cut it for Java 9,
> because the classification keyword / identifier cannot be made at
> the time when the stream passes the Lexer.

No, it's done between the lexer and the parser

> Let me remind you of this example:
>module foo { exports transitive
> How should the poor lexer recognize in this situation that transitive
> is an identifier (sic) (if you complete the text accordingly)?

There is a simple solution, consider module, requires, etc as keyword in the 
lexer, and when the keyword is sent to the parser, downgrade it to an 
identifier if you are not at the right dotted production.

It's easy to implement if your lexer/parser is non blocking, i.e if you push a 
bytebuffer to the lexer and the lexer push terminals to the parser. 

the other solution used by Tatoo is to instead of having one giant automata 
that recognize all tokens, works with a list automata that recognized each 
token and activate them or not depending on the parser state.

> Aside from specific heuristics
> (which are not available to any parser generator),

any but one :)

> we only know about this classification after the parser has matched
> an entire declaration.

so i suppose your parser is LR, you know the classification from the dotted 
production just before the terminal is about to be recognized.
when you construct the LR table, you know for each dotted production what are 
the terminals that can appear so the parser generator can keep these info in a 
side table and during the parsing, from the parser state, find which terminals 
can be recognized.

> I'm not even sure that theory has a name for this kind of grammar.
> Maybe we should speak of a constraint solver rather than a parser.

no need to have a constraint solver here, you need to export the terminals that 
will lead to a shift or a reduce for any LR states.  

> 
> Stephan
> 

Rémi

>>
>> Rémi
>>
>> [1] http://dl.acm.org/citation.cfm?id=1168057
>>
>> - Mail original -
>>> De: "Alex Buckley" 
>>> À: "Jayaprakash Arthanareeswaran" , "Dan Smith"
>>> , "Brian Goetz"
>>> 
>>> Cc: jigsaw-dev@openjdk.java.net
>>> Envoyé: Mercredi 3 Mai 2017 19:46:54
>>> Objet: Re: Java Platform Module System
>>
>>> On 5/2/2017 3:39 PM, Alex Buckley wrote:
>>>> On 5/2/2017 7:07 AM, Jayaprakash Arthanareeswaran wrote:
>>>>> Chapter 2 in [1] describes context-free grammars. The addition to "3.9
>>>>> Keywords" defines "restricted keywords", which prevent the grammar for
>>>>> ModuleDeclaration from being context-free. This prevents compilers from
>>>>> using common parser generators, since those typically only support
>>>>> context-free grammars. The lexical/syntactic grammar split defined in
>>>>> chapter 2 is not of much use for actual implementations of
>>>>> module-info.java parsers.
>>>>> The spec at least needs to point out that the given grammar for
>>>>> ModuleDeclaration is not actually context-free.
>>>>
>>>> The syntactic grammar in JLS8 was not context-free either; the opening
>>>> line of Chapter 2 has been false for years. For JLS9, I will remove the
>>>> claim that the lexical and syntactic grammars are context-free, and
>>>> perhaps a future JLS can discuss the difficulties in parsing the
>>>
>>> Jan Lahoda pointed out privately that the syntactic grammar in JLS8 and
>>> JLS9 is in fact context-free -- it's just not LL(1). Not being LL(1) is
>>> what I should have said the grammar hasn't been for a long time.
>>>
> >> Alex


Re: Java Platform Module System

2017-05-03 Thread Remi Forax
It's context-free because a context free grammar defined its input in term of 
terminals and the theory do not say how to map a token to a terminal.

Jay is right that it requires to use either some specific parser generator like 
Tatoo [1] the one i've written 10 years ago (because i wanted the tool to help 
me to extend a grammar easily) or to modify an existing parser generator so the 
parser can send the production state to the lexer which will enable/disable the 
automata that recognize the associated keywords .

Rémi

[1] http://dl.acm.org/citation.cfm?id=1168057

- Mail original -
> De: "Alex Buckley" 
> À: "Jayaprakash Arthanareeswaran" , "Dan Smith" 
> , "Brian Goetz"
> 
> Cc: jigsaw-dev@openjdk.java.net
> Envoyé: Mercredi 3 Mai 2017 19:46:54
> Objet: Re: Java Platform Module System

> On 5/2/2017 3:39 PM, Alex Buckley wrote:
>> On 5/2/2017 7:07 AM, Jayaprakash Arthanareeswaran wrote:
>>> Chapter 2 in [1] describes context-free grammars. The addition to "3.9
>>> Keywords" defines "restricted keywords", which prevent the grammar for
>>> ModuleDeclaration from being context-free. This prevents compilers from
>>> using common parser generators, since those typically only support
>>> context-free grammars. The lexical/syntactic grammar split defined in
>>> chapter 2 is not of much use for actual implementations of
>>> module-info.java parsers.
>>> The spec at least needs to point out that the given grammar for
>>> ModuleDeclaration is not actually context-free.
>>
>> The syntactic grammar in JLS8 was not context-free either; the opening
>> line of Chapter 2 has been false for years. For JLS9, I will remove the
>> claim that the lexical and syntactic grammars are context-free, and
>> perhaps a future JLS can discuss the difficulties in parsing the
> 
> Jan Lahoda pointed out privately that the syntactic grammar in JLS8 and
> JLS9 is in fact context-free -- it's just not LL(1). Not being LL(1) is
> what I should have said the grammar hasn't been for a long time.
> 
> Alex


Re: Accessing module internals from bytecode rewriting agent

2017-05-02 Thread Remi Forax
Hi Kirk,
--patch-module both at compile time and at runtime is what your are looking for.

cheers,
Rémi

- Mail original -
> De: "Kirk Pepperdine" 
> À: "Alan Bateman" 
> Cc: "jigsaw-dev" , core-libs-...@openjdk.java.net
> Envoyé: Mardi 2 Mai 2017 19:50:50
> Objet: Re: Accessing module internals from bytecode rewriting agent

> Hi Alan,
> 
> One of the techniques that I’ve used in the past for troubleshooting is to
> prepend modified classes to the bootstrap class path. I fear all this mucking
> about with modularization is going to leave the road to diagnostics littered
> with a ton of tools that will not longer work. As it is, 8 broke all the
> techniques and tooling used to diagnose class loader leaks… without fixing the
> fundamental problems that caused them to leak in the first place..and added a
> new sources for performance regressions for which there is little in the way 
> of
> tooling and certainly much less of an understanding of the fundamental issues
> out in the general developer population. The availability of high quality
> tooling is a huge bonus in the Java eco structure…. Please, don’t send us back
> to the Stone Age.
> 
> Kind regards,
> Kirk Pepperdine
> 
>> On May 2, 2017, at 9:48 AM, Alan Bateman  wrote:
>> 
>> On 02/05/2017 07:50, Jeremy Manson wrote:
>>> :
>>> 
>>> If we follow this path, before we migrate to Java 9, we would need to make 
>>> sure
>>> all of our code builds and the tests pass with Java 9 and Java 8.  We can't
>>> make all of the code build and the tests pass with Java 9 as-is, because 
>>> many
>>> of them use options like Xbootclasspath, which have been dropped.  We can't
>>> migrate those tests to use the new command line flags before we migrate to 
>>> Java
>>> 9, because if we do, they will stop working with Java 8.  Due to the size of
>>> our code base, it's pretty much impossible to migrate all of the users to a 
>>> new
>>> set of command line flags at the same time as we migrate to Java 9.
>> So I'm curious about -Xbootclasspath as that completely overrides the runtime
>> classes. Is this static instrumentation or combining the VM/launcher from one
>> build with the classes from a different JDK build? In any case, and as you 
>> have
>> found, -Xbootclasspath  and -Xbootclasspath/p are meaningless now, a
>> consequence of moving all the core APIs to modules. The -Xbootclasspath/a
>> option works as before with the caveat that not all platform classes are
>> visible to the boot class loader now (a consequence of ongoing security work 
>> to
>> move non-core components away from the fully privileged boot loader).
>> 
>> It might be that you mean the `javac -bootclasspath,  this works as before 
>> when
>> you target 8 or older.
>> 
>> -Alan


Re: Dependencies to a root modules when creating a dynamic module layer

2017-05-01 Thread forax
- Mail original -
> De: "Alan Bateman" 
> À: "Remi Forax" , "jigsaw-dev" 
> 
> Envoyé: Lundi 1 Mai 2017 13:41:30
> Objet: Re: Dependencies to a root modules when creating a dynamic module layer

> On 01/05/2017 10:48, Remi Forax wrote:
> 
>> Hi all,
>> i try to load a kind of dynamic plugin using a ModuleLayer,
>> the plugin is wrapped in a module which is loaded dynamically.
>> This module has a requires directive to "java.sql" which is part of the 
>> 'root'
>> modules.
>>
>> The problem is that my application do not requires "java.sql" because it do 
>> not
>> need it,
>> so "java.sql" is not part of the module loaded when the VM is booted,
>> then when i load the plugin module
>> - either it fails to find the required module java.sql [1]
>> - or i try to load java.sql from the ModuleLayer and get an exception because
>> java.sql can not be loaded by the boot classloader
>>
>>
>> I'm sure i've missed something ?
>>
> There isn't general support for dynamically augmenting the platform
> modules. There is partial support to allow the jdk.management.agent and
> java.instrument modules be dynamically loaded but that's all at this time.

In both cases, the VM adds modules to the configuration like with --add-modules 
before the boot layer is created, so there are not dynamically loaded.

> 
> One thing to point out is `--add-modules ALL-DEFAULT` in JEP 261. This
> is motivated by the scenario where a container needs to load
> applications that require additional modules.

I've decided to add a requires java.se because i know that no plugin will 
reference a module which is not present in java.se but like -with -add-modules 
ALL-DEFAULT, this is a kind of a workaround,
it's pre-loading all modules because you may need one.

I wonder if we can explore how to add modules to an existing configuration (if 
they were already accessible by the module finder used to create the 
configuration) in 10.

> 
> -Alan.

Rémi


Re: setAccessible() alternative with Jigsaw - feedback on Lookup

2017-04-26 Thread forax
- Mail original -
> De: "Matej Novotny" 
> À: "Remi Forax" 
> Cc: "Alan Bateman" , jigsaw-dev@openjdk.java.net, 
> "Tomas Remes" , "Martin
> Kouba" 
> Envoyé: Mercredi 26 Avril 2017 16:04:08
> Objet: Re: setAccessible() alternative with Jigsaw - feedback on Lookup

>> You can spin one module per package, by creating one ModuleLayer per package,
>> not unlike j.l.r.Proxy does.
> 
> Don't know a bit about it, but it sounds like a way worth exploring.
> Would you have a pointer towards some code showing how to deal with
> ModuleLayers?
> 
> Also checking on j.l.r.Proxy, it uses the "good old" Unsafe.defineClass.
> Can't really see any module-related magic there.

snap, sorry, the code has changed since the last time i looked at it, or it was 
another code ...

the recipe is:
  1. create a module with ModuleDescriptor.Builder [1]
 add the package you need with packages()
  2. create an implementation of ModuleFinder [2] that only find that module
  3. create a Configuration [3] like that resolve your module
 ModuleLayer boot = ModuleLayer.boot();
 Configuration cf = boot.configuration().resolve(finder, ModuleFinder.of(), 
moduleName);
  4. create a classloader 
 ClassLoader classLoader = new ClassLoader(parentClassLoader()) { /* make 
defineClass more visible */};
  5. create your own layer [4]
 ModuleLayer layer = boot.defineModulesWithOneLoader(cf, classloader;
  6. profit (i.e. now you can call define class on your classloader)

> 
> Matej

Rémi

[1] 
http://download.java.net/java/jdk9/docs/api/java/lang/module/ModuleDescriptor.Builder.html
[2] 
http://download.java.net/java/jdk9/docs/api/java/lang/module/ModuleFinder.html
[3] 
http://download.java.net/java/jdk9/docs/api/java/lang/module/Configuration.html#resolve-java.lang.module.ModuleFinder-java.lang.module.ModuleFinder-java.util.Collection-
[4] 
http://download.java.net/java/jdk9/docs/api/java/lang/ModuleLayer.html#defineModulesWithOneLoader-java.lang.module.Configuration-java.lang.ClassLoader-


Re: Alternatives for naming automatic modules, and a proposal (#AutomaticModuleNames)

2017-04-25 Thread Remi Forax
- Mail original -
> De: "mark reinhold" 
> À: "Brian Fox" 
> Cc: jigsaw-dev@openjdk.java.net
> Envoyé: Lundi 24 Avril 2017 20:54:18
> Objet: Re: Alternatives for naming automatic modules, and a proposal 
> (#AutomaticModuleNames)

> 2017/4/24 10:39:25 -0700, Brian Fox :
>> On Mon, Apr 24, 2017 at 12:14 PM, mark.reinh...@oracle.com wrote:
>>> The problem with any approach that allows you to give a stable name to
>>> what is otherwise an automatic module is that the API of that module is
>>> inherently unstable.  If and when the module is completely modularized
>>> then its API will almost certainly change, which in turn will require
>>> incompatible changes to any previously-published fully-modularized
>>> modules that depend upon it.
>> 
>> I don't quite follow, why does modularizing a module imply its public api
>> will change?
> 
> The effective API of a module is determined by the packages that it
> exports and, also, by the packages exported by the modules it depends
> upon, directly and indirectly, via `requires transitive` clauses.  The
> effective API of an existing library that's used as an automatic module
> is inherently unstable in two ways:
> 
>  - When you modularize the library explicitly, with a `module-info.java`
>file, you'll have to decide which packages to export.  Packages that
>are meant only for internal use should not be exported, but when your
>library was used as an automatic module those packages were exported.
>If you choose not to export those packages when you modularize then
>code in other modules that references types in those packages will
>break.
> 
>  - When you modularize the library explicitly you'll also have to decide
>which modules it requires transitively.  This will almost certainly
>further reduce the number of packages accessible to modules that use
>your module, and perhaps dramatically so: When your library was an
>automatic module it transitively required every other automatic
>module observable at compile time or run time, thus making every
>package of every automatic module accessible to modules that use your
>module.  Even worse, this set of packages depended upon the context
>in which the library was used; you can't compute it just by examining
>the library and its known dependencies.
> 
> An explicit module that depends upon one or more modules that are
> automatic today is, itself, no more stable than those automatic modules.
> It could be broken when those automatic modules are modularized
> explicitly, and if it `requires transitive` an automatic module then any
> modules that depend upon it could be broken when that automatic module
> is modularized explicitly.
> 
> If we define a way to give stable names to automatic modules, as the
> `Module-Name` manifest attribute would do, then that would tend to
> encourage the publication of named automatic modules and, further, the
> publication of explicit modules that depend upon them.  All of these
> modules would rest on shaky ground, if not actual quicksand.
> 
> If we encourage developers at large to take this approach then I worry
> that many will give up in frustration when they try to modularize their
> named automatic modules, later on, with explicit `module-info.java`
> files.  We'd wind up, in the long run, with public repositories that
> contain lots of named automatic modules but few actual explicit modules,
> when what we want to see is lots of explicit modules.  The best way to
> attain that long-term goal is to minimize the chances for pain and
> confusion along the way, and if that means a slower pace of migration
> then so be it.

It will be worst than that.

People want module resolution to be magic, it's what Maven or Gradle provide,
specifies your direct dependencies, we will care about the rest.

If build tools are not able to see existing plain old jars as modules, people 
will write plugins to work aroud that,
after all, Maven at its core is a tool that patch the deficiency of the 
classpath by linearizing the graph of dependencies,
so plugins will patch the non modular jars to make them modular and nobody will 
care how this is done.

In my opinion, having a tool that automatically modularize a bunch of jars is 
far worst that having authors of modules to delay the time to go full module.


Now about the stability of an automatic module. I don't think we have to care 
about that.
When migrating to the module world, either it's easy to wrap an existing code 
into a module and everybody is happy,
or it's hard and people will make a breaking change, create a new name (like 
common-collection4), and wrap the updated API into a new module.
Unlike with the JDK, as Brian said, people are used to upgrade (not update) 
their dependencies.


I believe that we should provide a way to be able to require a non modular jar 
with a stable name, so new project library or application will be able to use 
old jars as module rapidly,
oth

Re: Alternatives for naming automatic modules, and a proposal (#AutomaticModuleNames)

2017-04-25 Thread Remi Forax
- Mail original -
> De: "David M. Lloyd" 
> À: "Brian Fox" 
> Cc: "jigsaw-dev" 
> Envoyé: Mardi 25 Avril 2017 16:38:54
> Objet: Re: Alternatives for naming automatic modules, and a proposal 
> (#AutomaticModuleNames)


[...]

> 
> OSGi, by the way, avoids this situation somewhat more effectively than
> Jigsaw can, mainly because of three factors: resolving by package
> instead of by bundle name, performing the bulk of resolution work at run
> time, and by the simple fact that it is less invasive towards
> reflection, thus there is more "wiggle room".
> 
> This isn't to say that OSGi is an ideal solution of course, but I just
> wanted to point out why the OSGi ecosystem can generally get away with
> publishing artifacts containing descriptor information, while Jigsaw
> probably won't be able to (or at least, not as cleanly).

yes, 
you need a way to merge an artifact descriptor and its corresponding 
non-modular jar before the VM checks the modules configuration.

as you said, adding an assembly step after the modules being resolved and 
downloaded locally and before the VM is started is a solution.
At that point, you can either rename the automatic modules and put the internal 
dependencies of that module in the classpath or you can rewrite the module 
infos by doing a static analysis (+ some manual info because of the reflection).

> 
> --
> - DML

Rémi


Re: Alternatives for naming automatic modules, and a proposal (#AutomaticModuleNames)

2017-04-25 Thread Remi Forax
About multiple versions or true hiding of non exported packages, I do not 
believe that someone is against those features.

But they both requires to change the VM in an extensive way and to my knowledge 
nobody has ever done a prototype of those features.

The good news is that both these features can be introduced later in a 
compatible way.

I would love to see Oracle or RedHat to put engineering ressources on these 
features, but until that, the reality is that those features are not blockers 
to the release of JPMS*

Rémi 
* given the tumultuous past of the modules in Java, being able to release 
something even if not perfect, is already a victory.







On April 25, 2017 3:16:54 PM GMT+02:00, Michael Nascimento  
wrote:
>On Tue, Apr 25, 2017 at 10:10 AM, David M. Lloyd
>
>wrote:
>
>> Just the fact that there is the *very idea* of a "fit"/"non-fit" for
>JPMS
>> is sad though.  It should have been the ubiquitous thing that
>everyone was
>> expecting.  But denying multiple versions?  Blowing up on run time
>cycles?
>> Reneging on the idea of being the basis of Java EE 9? These are
>things that
>> people will not be expecting.
>
>
>Some extra feedback: I delivered a talk yesterday at QCon SP and people
>were very much disappointed with these limitations, and also the fact
>packages must be defined by a single module even if they are not
>exported,
>because it was considered nearly impossible to track in real world
>scenarios.
>
>Regards,
>Michael

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.


Re: Module naming restrictions

2017-04-25 Thread Remi Forax
I was thinking of (a), an automatic module should not allow Java keywords, 
apart all methods ModuleDescriptor.new*Module should checks that each component 
of the qualified name is not a Java keyword.

The idea is that all variants of ModuleDescriptor.new*Module implements the JLS 
semantics and ModuleDescriptor.read() implements the JVMS semantics.

Rémi

- Mail original -
> De: "Stephan Herrmann" 
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Mardi 25 Avril 2017 13:49:20
> Objet: Re: Module naming restrictions

> When you say "bug" do you mean
> (a) the name of that automatic module should avoid any Java keyword, or
> (b) Java keywords should be accepted as (part of) a module name?
> 
> Option (a) would require a change in the specification of ModuleFinder
> whereas (b) currently conflicts with JLS 7.7 which clearly states
>   A module name consists of one or more Java identifiers (§3.8) separated by 
> "."
>   tokens.
> 
> While neither (a) nor (b) is fulfilled, libraries with a Java keyword in their
> name
> simply cannot be used as automatic modules, right?
> 
> Stephan
> 
> 
> On 25.04.2017 09:50, Remi Forax wrote:
>> Seems to be a bug to me.
>>
>> Rémi
>>
>> - Mail original -
>>> De: "Rafael Winterhalter" 
>>> À: "jigsaw-dev" 
>>> Envoyé: Mardi 25 Avril 2017 09:12:04
>>> Objet: Module naming restrictions
>>
>>> Hello,
>>>
>>> I did another update of my libraries to support Java 9 and found out that
>>> Byte Buddy cannot easily be used as an automatic module. It is published as
>>> byte-buddy.jar to Maven Central where its automatic module name would be
>>> byte.buddy. This does however not compile due to "byte" being a keyword in
>>> Java. Here is what javac gives me:
>>>
>>> src/module-info.java:2: error:  expected
>>>requires byte.buddy;
>>>^
>>> 1 error
>>>
>>> I expect that this is not a common problem but not an uncommon one either.
>>> Looking for artifacts named after Java keywords in Maven Central, you can
>>> find quite a few.
>>>
>>> I do not know if this is technically feasible but I suggest that module
>>> names should not come with any restrictions due to the historic freedom in
>>> naming which would just cause confusion if some projects had to be renamed
>>> just to satisfy the module descriptors restrictions.
>>>
> >> Best regards, Rafael


Re: Accessing module internals from bytecode rewriting agent

2017-04-25 Thread Remi Forax
[...]

> 
> Meanwhile, Mandy Chung has kindly offered to look into the security
> considerations that are at play and come up with a less restrictive
> policy which enforces the security needs more accurately. At that point
> I will probably remove the fallback -- in part because I hope that by
> then both reflection and method handles will employ a common blacklist
> of inaccessible methods. I think it would make sense for them both to
> offer equivalent access and for that access to take security seriously.

I've seen this kind of fallback in different language runtimes, mine included,
having a "less blunt" checks would be great.

> 
> regards,
> 
> 
> Andrew Dinn

Rémi


Re: Getting the automatic module name of non-modular JAR

2017-04-25 Thread Remi Forax
If you want an Optional, you can use findFirst() on a stream,
  Optional ref = ModuleFinder.of( jar 
).findAll().stream().findFirst();

Rémi

- Mail original -
> De: "Gunnar Morling" 
> À: "Alan Bateman" 
> Cc: "jigsaw-dev" 
> Envoyé: Mardi 25 Avril 2017 09:10:45
> Objet: Re: Getting the automatic module name of non-modular JAR

> I see; thanks, Alan.
> 
> I wanted to avoid using a regex or similar, in order to make sure the
> JDK's own automatic naming rules are applied instead of
> "re-implementing" them. I was kinda hoping for a method like
> 
>Path jar = ...;
>Optional ref = ModuleReference.of( jar );
> 
> 
> 2017-04-25 8:49 GMT+02:00 Alan Bateman :
>> On 24/04/2017 21:23, Gunnar Morling wrote:
>>
>>> Hi,
>>>
>>> Given a non-modular JAR (e.g. represented as Path), what's the easiest
>>> way to obtain the automatic module name derived for this JAR?
>>
>> If you just want the name then it might be more efficient to do it with a
>> regular expression.
>>
>>>
>>> I found the following:
>>>
>>>  Path nonModularJar = ...;
>>>  String automaticModuleName = ModuleFinder.of( nonModularJar )
>>>  .findAll()
>>>  .iterator()
>>>  .next()
>>>  .descriptor()
>>>  .name();
>>>
>>> Is this the best I can do?
>>>
>>> More generally speaking, is using ModuleFinder with a single path the
>>> only way to obtain a ModuleReference/ModuleDescriptor for a specific
>>> JAR?
>>
>> Yes, ModuleFinder is the only way (it might be more succulent to use stream
>> + findFirst but that is just detail). If you are only interested in the name
>> then you could of course open the JAR file. If it contains module-info.class
>> then read it with ModuleDescriptor.read, otherwise use a regex to derive the
>> name.
>>
> > -Alan


Re: Module naming restrictions

2017-04-25 Thread Remi Forax
Seems to be a bug to me.

Rémi

- Mail original -
> De: "Rafael Winterhalter" 
> À: "jigsaw-dev" 
> Envoyé: Mardi 25 Avril 2017 09:12:04
> Objet: Module naming restrictions

> Hello,
> 
> I did another update of my libraries to support Java 9 and found out that
> Byte Buddy cannot easily be used as an automatic module. It is published as
> byte-buddy.jar to Maven Central where its automatic module name would be
> byte.buddy. This does however not compile due to "byte" being a keyword in
> Java. Here is what javac gives me:
> 
> src/module-info.java:2: error:  expected
>requires byte.buddy;
>^
> 1 error
> 
> I expect that this is not a common problem but not an uncommon one either.
> Looking for artifacts named after Java keywords in Maven Central, you can
> find quite a few.
> 
> I do not know if this is technically feasible but I suggest that module
> names should not come with any restrictions due to the historic freedom in
> naming which would just cause confusion if some projects had to be renamed
> just to satisfy the module descriptors restrictions.
> 
> Best regards, Rafael


Re: setAccessible() alternative with Jigsaw - feedback on Lookup

2017-04-24 Thread Remi Forax


- Mail original -
> De: "Matej Novotny" 

> Hi,
> 
> thanks for your time. Comments inline.
> 

[...]

>> 
>> If you are looking to define a class in a new runtime package and the
>> class loader that is not your implementation then you are out of luck.
>> Lookup is designed for doing computations in the context of an existing
>> class. There is a note in the archives here from John Rose about
>> extending the lookup API for cases like this but it would come with a
>> lot of complexity. Also there isn't any notion of adding a package to a
>> named module at this time. When java.lang.reflect.Proxy spins proxy
>> classes then it doesn't add packages to named modules, it instead spins
>> the proxy classes into its own module (something you could do too,
>> assuming the classes that you are proxying are accessible).
> 
> I think it's more or less what we did/are doing - we spin the proxies either
> into the same package the came from or to our own package/module.
> But in order to avoid proxy name clash in the same package, we just took the
> original package and replaced the prefix 'java' with something like
> 'org.jboss.weld.proxies'.
> This effectively means that 'java.util.Map' proxy will end up as something 
> like
> 'org.jboss.weld.proxies.util.Map$Proxy'.
> Obviously, 'org.jboss.weld.proxies.util' did not exist beforehand and putting
> everything into one existing package using Lookup is bound to blow up very
> quickly.
> So there is basically no way to achieve this with JDK 9?

You can spin one module per package, by creating one ModuleLayer per package, 
not unlike j.l.r.Proxy does.

Rémi

> 
>> 
>> >
>> > 3) All in all the changes carried along a lot of complexity compared to
>> > plain old ClassLoader.defineClass
>> > We need to create A LOT of Lookups (might present performance overhead,
>> > haven't tested yet).
>> > Extra care needs to be given to the packages for Lookup not to blow up
>> > while we in fact don't really care where the proxy lands as long as it is
>> > usable.
>> Can you expand on the "don't really care" comment? Do you mean that you
>> don't care about either the defining loader or runtime package?
> 
> I was referring to the runtime package.
> What we need is to ensure that there won't be a clash - there can be two 
> classes
> with same name and different packages, which, for some reason, will have to
> land in a package created by us.
> If this happened and we were using a fixed existing package to place them in, 
> it
> would blow up.
> So what I meant was that with JDK 8 I don't need to care for this, I just use
> prefix and I am done with it. In JDK 9, this seems like quite an obstacle.
> 
>> 
>> >
>> >
>> > Another nasty thing is that the code of course needs to work with both, JDK
>> > 9 and 8.
>> > While it isn't impossible, it will add a not-so-nice reflection magic layer
>> > to the mix.
>> >
>> Multi-release JARs (JEP 238 [3]) and the ability to compile to older
>> releases (JEP 247 [4]) might be useful here (you might know about these
>> features already).
> 
> Haven't really explored this yet, it might be a way.
> Gotta see how Maven deals with this feature to allow us to create MRJARs in a
> reasonable manner.
> 
> Matej
> 
>> 
>> -Alan.
>> 
>> [1] https://bugs.openjdk.java.net/browse/JDK-8171335
>> [2] http://openjdk.java.net/jeps/181
>> [3] http://openjdk.java.net/jeps/238
>> [4] http://openjdk.java.net/jeps/247


Re: Changing the default file system provider is broken :(

2017-04-23 Thread Remi Forax
- Mail original -
> De: "Alan Bateman" 
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Dimanche 23 Avril 2017 12:38:21
> Objet: Re: Changing the default file system provider is broken :(

> On 22/04/2017 21:48, Peter Levart wrote:
> 
>>
>> Well, it works at least when
>> -Djava.nio.file.spi.DefaultFileSystemProvider=... is specified on the
>> command line. But when the system property is set as 1st thing in
>> main() method, it is already too late. It seems that after boot
>> sequence is finished (VM.isBooted() == true)and before main() method
>> is called, some code needs the FileSystem again, so default file
>> system is already initialized before the main() is executed.
>>
>> Perhaps there needs to be another initLevel which would be set just
>> before the invocation of the main() method?
> I don't think we go there as there are just too many scenarios where
> "user code" can execute before the application main method. Instead, I
> think this is a case where the default file system provider can only be
> overridden by setting the property on the command line.

This should be added to the compatibility list because mocking the FileSystem 
for testing purpose is common, at least in my own bubble.

> 
> -Alan

Rémi


Re: Changing the default file system provider is broken :(

2017-04-22 Thread forax
[...] 

> Or this might work too:

> http://cr.openjdk.java.net/~plevart/jdk9-dev/Path.toFile/webrev.01/
yes, it's better 

> A variant of this might be for FileSystems.getDefault() to start returning the
> custom default filesystem as soon as initLevel reaches 2 (after module system
> is initialized). This might allow custom default file system to be effective
> even before the VM is fully booted (for custom security manager or system 
> class
> loader or java agent to already take advantage of it). But that probably would
> mean that the custom default filesystem loading logic would have to be 
> modified
> in order to cope with system class loader not being setup already - it could
> use the builtin app class loader for locating and loading the classes...

> Do you happen to have a custom default filesystem to try this with?
sure, a dummy one that delegates everything to the builtin 
https://gist.github.com/forax/615828b41c33eccda33afdf2eba1db87 

> Regards, Peter
regards, 
Rémi 


Re: Changing the default file system provider is broken :(

2017-04-22 Thread forax
Hi Peter 

> Hi Remi,

> On 04/22/2017 01:20 PM, Remi Forax wrote:

>> Ooops !
>> if you use the module path, you can not change the default system provider
>> mechanism anymore,
>> mostly because the module implementation internally uses java.nio.file.Path 
>> at
>> different places.

>> First, trying to initialize the default system provider in Java code do not 
>> work
>> at all,
>> calling
>>   System.setProperty("java.nio.file.spi.DefaultFileSystemProvider", "...");
>> in a main does nothing because the jigsaw implementation has already loaded 
>> the
>> default implementation to be able to inspect the modules.

>> Then, if you try to use the command line
>>  -Djava.nio.file.spi.DefaultFileSystemProvider=...
>> you have various bootstrapping issues.

>> With exploded modules, when trying to initialize the default file system, the
>> part of jigsaw that reads exploded modules uses path.toFile() that fails it
>> checks that the path belongs to the default file system which is not already
>> initialized
>> (see below for the whole stacktrace)

>> With modular jars, when trying to initialize the default file system, the
>> JarModuleReader also uses path.toFile(), leading to the same error

>> One way to solve that is to change the code in FileSystems.getDefault() to 
>> add a
>> nullcheck,
>> FileSystem defaultFileSystem = 
>> DefaultFileSystemHolder.defaultFileSystem;
>> if (jdk.internal.misc.VM.isBooted() && defaultFileSystem != null) {
>> return defaultFileSystem;
>> } else {
>> return BuiltinFileSystemHolder.builtinFileSystem;
>> }
>> Another solution is to change the code of jigsaw to only use the
>> builtinFileSystem, delaying the initialization of the default system which 
>> will
>> also solve the case where an application uses System.setProperty.

> I think the problem is that what FileSystems.getDefault() returns, changes 
> over
> time. So you have Path instances associated with "builtin" filesystem,
> constructed before VM.isBooted() and after it is booted, Path instances are
> associated with what is configured to be the "default" filesystem. If those 
> two
> differ and an instance, constructed before VM.isBooted() lives past the boot
> time and then .toFile() is called on such instance, we get
> "UnsupportedOperationException: Path not associated with default file
> system"...

> Maybe Alan could shed some light into this. As part of jigsaw implementation,
> the FileSystems.getDefault() was changed from:

> public static FileSystem getDefault() {
> return DefaultFileSystemHolder.defaultFileSystem;
> }

> to:

> public static FileSystem getDefault() {
> if (jdk.internal.misc.VM.isBooted()) {
> return DefaultFileSystemHolder.defaultFileSystem;
> } else {
> return BuiltinFileSystemHolder.builtinFileSystem;
> }
> }

> Probably because of early bootstrap issues. Module system needs FileSystem and
> custom FileSystem classes can not be loaded at that time yet.

> So I think that your second solution would be the right one. Jigsaw should 
> only
> use the builtin filesystem explicitly, refraining from initializing the 
> default
> filesystem. And Path.toFile() should allow creating a File from a Path in
> either case: whether Path is associated with builtin or default filesystem.
I think your right for the modular jars case, ModulePath store an array of Path 
created before the VM is considered as booted and consume them lazily :( 
I believe that replacing the path.toFile() by new File(path.toString()) will 
fix this issue. 

> Regards, Peter
Rémi 

>> cheers,
>> Rémi

>> Stacktrace with exploded modules:
>> Error: A JNI error has occurred, please check your installation and try again
>> Exception in thread "main" java.lang.Error:
>> java.lang.UnsupportedOperationException: Path not associated with default 
>> file
>> system.
>>  at
>>  
>> java.base/java.nio.file.FileSystems$DefaultFileSystemHolder.getDefaultProvider(FileSystems.java:139)
>>  at
>>  
>> java.base/java.nio.file.FileSystems$DefaultFileSystemHolder.access$100(FileSystems.java:100)
>>  at
>>  
>> java.base/java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:109)
>>  at
>>  
>> java.base/java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:107)
>>  at java.base/java.security.AccessController.doPrivileged(Native Method)
>>  at
>>  
>> java.base/java.

Changing the default file system provider is broken :(

2017-04-22 Thread Remi Forax
Ooops !
if you use the module path, you can not change the default system provider 
mechanism anymore,
mostly because the module implementation internally uses java.nio.file.Path at 
different places.

First, trying to initialize the default system provider in Java code do not 
work at all,
calling
  System.setProperty("java.nio.file.spi.DefaultFileSystemProvider", "...");
in a main does nothing because the jigsaw implementation has already loaded the 
default implementation to be able to inspect the modules.

Then, if you try to use the command line
 -Djava.nio.file.spi.DefaultFileSystemProvider=...
you have various bootstrapping issues.

With exploded modules, when trying to initialize the default file system, the 
part of jigsaw that reads exploded modules uses path.toFile() that fails it 
checks that the path belongs to the default file system which is not already 
initialized
(see below for the whole stacktrace)

With modular jars, when trying to initialize the default file system, the 
JarModuleReader also uses path.toFile(), leading to the same error

One way to solve that is to change the code in FileSystems.getDefault() to add 
a nullcheck,
FileSystem defaultFileSystem = 
DefaultFileSystemHolder.defaultFileSystem;
if (jdk.internal.misc.VM.isBooted() && defaultFileSystem != null) {
return defaultFileSystem;
} else {
return BuiltinFileSystemHolder.builtinFileSystem;
}
Another solution is to change the code of jigsaw to only use the 
builtinFileSystem, delaying the initialization of the default system which will 
also solve the case where an application uses System.setProperty.

cheers,
Rémi


Stacktrace with exploded modules:
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.Error: 
java.lang.UnsupportedOperationException: Path not associated with default file 
system.
at 
java.base/java.nio.file.FileSystems$DefaultFileSystemHolder.getDefaultProvider(FileSystems.java:139)
at 
java.base/java.nio.file.FileSystems$DefaultFileSystemHolder.access$100(FileSystems.java:100)
at 
java.base/java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:109)
at 
java.base/java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:107)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at 
java.base/java.nio.file.FileSystems$DefaultFileSystemHolder.defaultFileSystem(FileSystems.java:107)
at 
java.base/java.nio.file.FileSystems$DefaultFileSystemHolder.(FileSystems.java:101)
at java.base/java.nio.file.FileSystems.getDefault(FileSystems.java:188)
at java.base/java.nio.file.Path.toFile(Path.java:655)
at 
java.base/jdk.internal.module.ModuleReferences$JarModuleReader.newJarFile(ModuleReferences.java:229)
at 
java.base/jdk.internal.module.ModuleReferences$JarModuleReader.(ModuleReferences.java:239)
at 
java.base/jdk.internal.module.ModuleReferences.lambda$newJarModule$0(ModuleReferences.java:96)
at 
java.base/jdk.internal.module.ModuleReferenceImpl.open(ModuleReferenceImpl.java:88)
at 
java.base/jdk.internal.loader.BuiltinClassLoader.createModuleReader(BuiltinClassLoader.java:953)
at 
java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1719)
at 
java.base/jdk.internal.loader.BuiltinClassLoader.moduleReaderFor(BuiltinClassLoader.java:945)
at 
java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:702)
at 
java.base/jdk.internal.loader.BuiltinClassLoader.findClassInModuleOrNull(BuiltinClassLoader.java:651)
at 
java.base/jdk.internal.loader.BuiltinClassLoader.findClass(BuiltinClassLoader.java:532)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:585)
at java.base/java.lang.Class.forName(Class.java:447)
at 
java.base/sun.launcher.LauncherHelper.loadModuleMainClass(LauncherHelper.java:585)
at 
java.base/sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:523)
Caused by: java.lang.UnsupportedOperationException: Path not associated with 
default file system.
at java.base/java.nio.file.Path.toFile(Path.java:658)
at 
java.base/jdk.internal.module.ModuleReferences$JarModuleReader.newJarFile(ModuleReferences.java:229)
at 
java.base/jdk.internal.module.ModuleReferences$JarModuleReader.(ModuleReferences.java:239)
at 
java.base/jdk.internal.module.ModuleReferences.lambda$newJarModule$0(ModuleReferences.java:96)
at 
java.base/jdk.internal.module.ModuleReferenceImpl.open(ModuleReferenceImpl.java:88)
at 
java.base/jdk.internal.loader.BuiltinClassLoader.createModuleReader(BuiltinClassLoader.java:953)
at 
java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1719)
at 
java.base/jdk.internal.loader.BuiltinC

Re: Review Request JDK-8177844: Ensure non-upgradeable modules cannot be upgraded even with --patch-module

2017-04-08 Thread Remi Forax
Ok,
thanks.

Rémi 

On April 8, 2017 4:49:13 PM GMT+02:00, Mandy Chung  
wrote:
>No change in that regards.  The updated test [1] in the webrev shows it
>patches java.base/jdk.internal.modules.SystemModules
>
>Before this fix, you can do —patch-module jdk.internal.vm.compiler=.jar
>—upgrade-module-path graal.jar. The reason is that —patch-module
>disables the hash checking that makes non-upgradeable module
>upgradeable.  After this fix, it fails if you attempt upgrade a patched
>non-upgradeable module.
>
>[1]
>http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8177844/webrev.01/test/tools/launcher/modules/patch/systemmodules/PatchSystemModules.java.frames.html
>
>> On Apr 8, 2017, at 7:22 AM, Remi Forax  wrote:
>> 
>> Hi Mandy,
>> how can I test a change in java.lang after that patch ?
>> 
>> Rémi
>> 
>> 
>> On April 8, 2017 7:50:50 AM GMT+02:00, Mandy Chung
> wrote:
>> http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8177844/webrev.01
><http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8177844/webrev.01>/
>> 
>> This fixes -—patch-module to do hash checking on the module being
>patched
>> so that it will ensure that a non-upgradeable module remains not
>upgradeable.
>> 
>> As Graal has been using —-patch-module option to disable the hash
>checking
>> to load a different version of jdk.internal.vm.compiler, it needs a 
>> mechanism to load Graal, as tracked by JDK-8177845.
>> 
>> Mandy
>> 
>> -- 
>> Sent from my Android device with K-9 Mail. Please excuse my brevity.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.


Re: Review Request JDK-8177844: Ensure non-upgradeable modules cannot be upgraded even with --patch-module

2017-04-08 Thread Remi Forax
Hi Mandy,
how can I test a change in java.lang after that patch ?

Rémi


On April 8, 2017 7:50:50 AM GMT+02:00, Mandy Chung  
wrote:
>http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8177844/webrev.01/
>
>This fixes -—patch-module to do hash checking on the module being
>patched
>so that it will ensure that a non-upgradeable module remains not
>upgradeable.
>
>As Graal has been using —-patch-module option to disable the hash
>checking
>to load a different version of jdk.internal.vm.compiler, it needs a 
>mechanism to load Graal, as tracked by JDK-8177845.
>
>Mandy

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.


Re: Disallowing the dynamic loading of agents by default (revised)

2017-04-06 Thread Remi Forax
- Mail original -
> De: "mark reinhold" 
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Mercredi 5 Avril 2017 18:15:20
> Objet: Disallowing the dynamic loading of agents by default (revised)

> Thanks to everyone for the quick feedback on this topic, and especially
> to Andrew for the constructive dialogue.
> 
> Here's a revised proposal:
> 
>  - Define a new VM option, `-XX:+EnableDynamicAgentLoading`, that's
>on by default in JDK 9 but off by default in JDK 10.
> 
>This will allow launch scripts that use this option on JDK 10 to
>work on JDK 9 without change, and will allow early testing of the
>JDK 10 behavior on JDK 9.

yes


> 
>  - Revise the `com.sun.tools.attach` API to forbid attachment to the
>current process or to an ancestor of the current process, and
>define a read-only system property that allows such attachment to
>be enabled via the command line.
> 
>This will discourage the inadvertent use of libraries that, for
>better or for worse, intentionally violate strong encapsulation.

don't get this one, as David said, if you span a new VM with an exec, you have 
more right ??


> 
>  - Enhance the `-jar` launcher option so that if the JAR file being
>launched contains a `Premain-Class` attribute then it's launched
>as both an application and as an agent for that application.
> 
>This will allow `java -jar foo.jar` to be used in place of the
>more verbose `java -javaagent:foo.jar -jar foo.jar` [1].


Can be very useful indeed.
(with another name that "Premain-Class" for backward compatibility).

> 
> Taken together, these changes are intended to enable the continued use
> of legitimate dynamically-loaded agents without change on JDK 9 and with
> a small change on JDK 10.  That later change will align the treatment of
> such agents with the other means of breaking encapsulation (`--add-opens`,
> etc.) in order to ensure integrity by default for all code.
> 
> This proposal does not attempt to lock down platform classes as distinct
> from user classes.  Many agents have legitimate reasons to transform
> platform classes, so an additional mechanism to protect those classes
> does not appear to be worthwhile.
> 
> Comments?
> 
> - Mark
> 
> 
> [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012000.html

Rémi


Re: Progress report on SLF4J project modularization

2017-03-26 Thread forax
- Mail original -
> De: "Robert Scholte" 
> À: "Remi Forax" 
> Cc: jigsaw-dev@openjdk.java.net
> Envoyé: Dimanche 26 Mars 2017 11:46:40
> Objet: Re: Progress report on SLF4J project modularization

> Hi Rémi,

caveat: i'm not fully competent here,

> 
> IIUC the only way to confirm that all requirements are added to the module
> descriptor is to compile the module-info file together with its sources. A
> correct and complete module-info is the base of the modular system.
> That's the reason why I chose for the first option.

I believe that if you are compiling the module-info.java with the already 
compiled classes, it should work (apart if you compile with 
--module-source-path)
I fail to see what check the compiler will not done in that case.

> 
> thanks,
> Robert

Rémi

> 
> On Sat, 25 Mar 2017 21:32:18 +0100, Remi Forax  wrote:
> 
>> Robert,
>> they are two ways to compile a project with a version which is not java
>> 9, let say 8 and a module-info with java 9.
>>
>> Either, like the Maven compiler plugin, you compile twice the source,
>> once with java 9 and once with java 8 excluding the module-info.java,
>> or you compile once with java 8 excluding the module-info and you then
>> compile only the module-info.java with java 9 (it's what Ceki has used).
>>
>> The later solution is usually faster and avoid subtle bugs like if
>> between java x and java 9, the code of javac changes the way a synthetic
>> class is named (you will have the two versions with the compiler plugin).
>>
>> cheers,
>> Rémi
>> - Mail original -
>>> De: "Robert Scholte" 
>>> À: jigsaw-dev@openjdk.java.net
>>> Envoyé: Samedi 25 Mars 2017 18:56:50
>>> Objet: Re: Progress report on SLF4J project modularization
>>
>>> On Sat, 25 Mar 2017 15:38:27 +0100, Ceki Gulcu  wrote:
>>>
>>>> Following the recipe at [2], it all seems to work.
>>>
>>> Good to read that the recipe works. If we need to add things to this
>>> recipe to improve it, just let me know.
>>> Although in general I don't think we'll be helping developers if we show
>>> the javac with its commandline arguments, especially since most of it is
>>> hidden or calculated inside the maven-compiler-plugin.
>>>
>>> thanks,
> >> Robert


Re: Progress report on SLF4J project modularization

2017-03-25 Thread Remi Forax
Robert,
they are two ways to compile a project with a version which is not java 9, let 
say 8 and a module-info with java 9.

Either, like the Maven compiler plugin, you compile twice the source, once with 
java 9 and once with java 8 excluding the module-info.java,
or you compile once with java 8 excluding the module-info and you then compile 
only the module-info.java with java 9 (it's what Ceki has used).

The later solution is usually faster and avoid subtle bugs like if between java 
x and java 9, the code of javac changes the way a synthetic class is named (you 
will have the two versions with the compiler plugin).

cheers,
Rémi
 
- Mail original -
> De: "Robert Scholte" 
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Samedi 25 Mars 2017 18:56:50
> Objet: Re: Progress report on SLF4J project modularization

> On Sat, 25 Mar 2017 15:38:27 +0100, Ceki Gulcu  wrote:
> 
>> Following the recipe at [2], it all seems to work.
> 
> Good to read that the recipe works. If we need to add things to this
> recipe to improve it, just let me know.
> Although in general I don't think we'll be helping developers if we show
> the javac with its commandline arguments, especially since most of it is
> hidden or calculated inside the maven-compiler-plugin.
> 
> thanks,
> Robert


Re: Better tools for adjusting to strong encapsulation

2017-03-22 Thread Remi Forax
good to me.

Rémi


On March 23, 2017 4:30:00 AM GMT+01:00, mark.reinh...@oracle.com wrote:
>Thanks to everyone for all the feedback on this topic.
>
>It appears that issuing warning messages for illegal-access operations
>enabled by the precise `--add-opens` and `--add-exports` options is a
>bit too aggressive, at least for JDK 9.  Perhaps we can enable that in
>JDK 10 after there's been more time for libraries, frameworks, and even
>the JDK itself to adjust to the realities of strong encapsulation.
>
>For now I suggest that we revert to the previous behavior of these two
>options, so that they do not cause warning messages to be issued.  The
>new `--permit-illegal-access` option will continue to generate warning
>messages, as proposed.  If those messages are a problem in a particular
>scenario then they can be eliminated by switching to an appropriate set
>of `--add-opens` options, which can be constructed from the information
>contained in those messages.
>
>Comments?
>
>- Mark

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.


Re: 8174823: Module system implementation refresh (3/2017)

2017-03-22 Thread Remi Forax
So ModuleTarget.osVersion is removed, i will have to update ASM accordingly.

In jdk/internal/module/ModuleBootstrap.java,
  addExtraExportsOrOpens(bootLayer, extraOpens, true, builder);
should be
  addExtraExportsOrOpens(bootLayer, extraOpens, ADD_OPENS, builder);
there is no point to introduce a boolean here.

In java.lang.reflect.Method,
the StringJoiner is created even if there are no parameters.

AccessibleObject.toShortString() should have a javadoc.

Rémi

- Mail original -
> De: "Alan Bateman" 
> À: "jigsaw-dev" , 
> "hotspot-runtime-...@openjdk.java.net runtime"
> 
> Envoyé: Mardi 21 Mars 2017 22:42:02
> Objet: 8174823: Module system implementation refresh (3/2017)

> We have been accumulating changes in the jake forest for the last few
> weeks to align with the proposals in JSR 376 and also to pick up API
> changes and bug fixes. It's time to bring these changes into jdk9/dev.
> The issue to bring these changes into jdk9/dev in bulk has been approved
> via the FC extension process (still in use during JDK 9 RDP2).
> 
> A summary of the main changes is listed in JDK-8174823 [1], and the
> webrevs with the changes to bring to jdk9/dev are here:
>http://cr.openjdk.java.net/~alanb/8174823/1/
> 
> The changes are currently based on jdk-9+161 and will be rebased before
> pushing.
> 
> -Alan
> 
> [1] https://bugs.openjdk.java.net/browse/JDK-8174823


  1   2   3   4   >