Re: Loading providers in named modules with ServiceLoader using a plugin class realm

2017-06-26 Thread Guillaume Boué 
From what I tested, ServiceLoader doesn't allow a custom classloading 
scheme for named modules (step 1 of 
http://download.java.net/java/jdk9/docs/api/java/util/ServiceLoader.html#load-java.lang.Class-java.lang.ClassLoader-). 
Its own scheme relies on checking modules loaded by the given 
classloader, then its parents (with a particular rule with regard to 
module layers), and it doesn't look like customizable. It also seems 
that existing META-INF/services declarations, but located in named 
modules, are ignored. For unnamed modules, or when something other than 
ServiceLoader is used, everything is apparently the same as before in 
Java 9.


This is why I think a plugin wouldn't be able to locate providers in a 
module loaded by Maven core extension classloader: parent of the plugin 
realm is bootstrap classloader directly.


Guillaume


Le 26/06/2017 à 01:45, Igor Fedorenko a écrit :

Can you explain little more why plugins won't see classes loaded by
maven core or maven core extensions classloaders? This is implemented in
classwords and I was under impression that java9 still allowed custom
classloading schemes like what we do or like what OSGi does.




---
L'absence de virus dans ce courrier électronique a été vérifiée par le logiciel 
antivirus Avast.
https://www.avast.com/antivirus


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Loading providers in named modules with ServiceLoader using a plugin class realm

2017-06-25 Thread Igor Fedorenko 
Can you explain little more why plugins won't see classes loaded by
maven core or maven core extensions classloaders? This is implemented in
classwords and I was under impression that java9 still allowed custom
classloading schemes like what we do or like what OSGi does.

-- 
Regards,
Igor

On Sun, Jun 25, 2017, at 07:08 PM, Guillaume Boué wrote:
> 
> Le 25/06/2017 à 22:03, Chas Honton a écrit :
> > Under what circumstances would a plugin not want the platform classloader?
> >
> > Chas
> 
> Thinking about this more, with the current situation, it actually means 
> that potential providers, located in named modules loaded for example by 
> Maven core classloader or extension classloader, will not be available 
> to plugins using ServiceLoader. So this is bigger than platform 
> classloader. As Robert said, I think there needs to be a way for plugins 
> to know about modules.
> 
> Reading more into the docs, this may be possible using a ModuleLayer. 
> Each plugin would have its own module layer composed of modules found in 
> its dependencies. Their parent would be a module layer associated to 
> Maven core, and its parent would be the boot layer. This would solve the 
> ServiceLoader issue for named modules in a plugin realm, since it would 
> locate providers in all modules in the module layer of the plugin, and 
> then do the same for their parents, up to the boot layer. However, for 
> it to work, I'm not sure if this implies that everything be made modular 
> (plugins and Maven itself). I will try to do some testing on this.
> 
> Guillaume
> 
> >
> >> On Jun 25, 2017, at 12:40 PM, Robert Scholte  wrote:
> >>
> >> Hi Guillaume,
> >>
> >> I don't know all the details about the Platform classloader, but it has 
> >> been introduced with a reason.
> >> So I don't think we should switch to it by default. I think the plugin is 
> >> well aware which classloaders / modules it wants to use (it should be), so 
> >> I think we need to find a mechanism for plugins to select their 
> >> classloaders and modules.
> >>
> >> thanks,
> >> Robert
> >>
> >>> On Sun, 25 Jun 2017 17:19:07 +0200, Guillaume Boué  
> >>> wrote:
> >>>
> >>> Hi,
> >>>
> >>> With the introduction of modules in JDK 9, there were changes with
> >>> regard to how classloading works, and this impacts class realms created
> >>> in Maven. Today, the parent (as per ClassLoader.getParent()) of a class
> >>> realm is null, which represents the bootstrap classloader. In JDK 9, the
> >>> change is that some classes were moved to a named module other than
> >>> java.base, and they are not loaded with the bootstrap classloader
> >>> anymore, but with the platform classloader (which was previously the
> >>> extension classloader, see JDK-814637).
> >>>
> >>> This has consequences, like MANTRUN-200, where locating providers with
> >>> the ServiceLoader API, using the plugin class realm, will miss JDK
> >>> internal implementation classes. In the case of MANTRUN-200, it is
> >>> Nashorn of the jdk.scripting.nashorn module, that cannot be found
> >>> because of the way ServiceLoader works
> >>> http://download.java.net/java/jdk9/docs/api/java/util/ServiceLoader.html#load-java.lang.Class-java.lang.ClassLoader-.
> >>> During the search in named modules, the class realm will not be able to
> >>> use its strategy since that process relies on parent delegation
> >>> implemented as explicit calls to ClassLoader.getParent()
> >>> (http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/f3cf7fd26baa/src/java.base/share/classes/java/util/ServiceLoader.java#l1062),
> >>> which, for a class realm, corresponds to the base classloader, i.e. the
> >>> bootstrap classloader. And since the Nashorn script engine factory was
> >>> loaded with the platform classloader, it is missed.
> >>>
> >>> It seems that the fix here would be to make all class realms have as
> >>> base classloader the platform classloader starting with JDK 9, instead
> >>> of the bootstrap classloader (there is a new utility method in
> >>> ClassLoader to obtain the platform classloader). I verified that this
> >>> solves the problem described in MANTRUN-200, but before I create an MNG
> >>> issue, I'm wondering if this is the correct approach.
> >>>
> >>> What do you think of this change to class realms? The other possibility
> >>> I can think of would be to have a way in the JDK to override the search
> >>> in named modules, so that our ClassRealm can also delegate to its parent
> >>> classloader. It is possible for unnamed modules (since the search
> >>> process then relies on the ClassLoader.getResources method, that can be
> >>> overriden) but it doesn't look like (and probably intentionally so) to
> >>> be possible for named modules.
> >>>
> >>> Guillaume
> >>>
> >>>
> >>> ---
> >>> L'absence de virus dans ce courrier électronique a été vérifiée par le 
> >>> logiciel antivirus Avast.
> >>> https://www.avast.com/antivirus
> >>>
> >>>
> >>>

Re: Loading providers in named modules with ServiceLoader using a plugin class realm

2017-06-25 Thread Guillaume Boué 


Le 25/06/2017 à 22:03, Chas Honton a écrit :

Under what circumstances would a plugin not want the platform classloader?

Chas


Thinking about this more, with the current situation, it actually means 
that potential providers, located in named modules loaded for example by 
Maven core classloader or extension classloader, will not be available 
to plugins using ServiceLoader. So this is bigger than platform 
classloader. As Robert said, I think there needs to be a way for plugins 
to know about modules.


Reading more into the docs, this may be possible using a ModuleLayer. 
Each plugin would have its own module layer composed of modules found in 
its dependencies. Their parent would be a module layer associated to 
Maven core, and its parent would be the boot layer. This would solve the 
ServiceLoader issue for named modules in a plugin realm, since it would 
locate providers in all modules in the module layer of the plugin, and 
then do the same for their parents, up to the boot layer. However, for 
it to work, I'm not sure if this implies that everything be made modular 
(plugins and Maven itself). I will try to do some testing on this.


Guillaume




On Jun 25, 2017, at 12:40 PM, Robert Scholte  wrote:

Hi Guillaume,

I don't know all the details about the Platform classloader, but it has been 
introduced with a reason.
So I don't think we should switch to it by default. I think the plugin is well 
aware which classloaders / modules it wants to use (it should be), so I think 
we need to find a mechanism for plugins to select their classloaders and 
modules.

thanks,
Robert


On Sun, 25 Jun 2017 17:19:07 +0200, Guillaume Boué  wrote:

Hi,

With the introduction of modules in JDK 9, there were changes with
regard to how classloading works, and this impacts class realms created
in Maven. Today, the parent (as per ClassLoader.getParent()) of a class
realm is null, which represents the bootstrap classloader. In JDK 9, the
change is that some classes were moved to a named module other than
java.base, and they are not loaded with the bootstrap classloader
anymore, but with the platform classloader (which was previously the
extension classloader, see JDK-814637).

This has consequences, like MANTRUN-200, where locating providers with
the ServiceLoader API, using the plugin class realm, will miss JDK
internal implementation classes. In the case of MANTRUN-200, it is
Nashorn of the jdk.scripting.nashorn module, that cannot be found
because of the way ServiceLoader works
http://download.java.net/java/jdk9/docs/api/java/util/ServiceLoader.html#load-java.lang.Class-java.lang.ClassLoader-.
During the search in named modules, the class realm will not be able to
use its strategy since that process relies on parent delegation
implemented as explicit calls to ClassLoader.getParent()
(http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/f3cf7fd26baa/src/java.base/share/classes/java/util/ServiceLoader.java#l1062),
which, for a class realm, corresponds to the base classloader, i.e. the
bootstrap classloader. And since the Nashorn script engine factory was
loaded with the platform classloader, it is missed.

It seems that the fix here would be to make all class realms have as
base classloader the platform classloader starting with JDK 9, instead
of the bootstrap classloader (there is a new utility method in
ClassLoader to obtain the platform classloader). I verified that this
solves the problem described in MANTRUN-200, but before I create an MNG
issue, I'm wondering if this is the correct approach.

What do you think of this change to class realms? The other possibility
I can think of would be to have a way in the JDK to override the search
in named modules, so that our ClassRealm can also delegate to its parent
classloader. It is possible for unnamed modules (since the search
process then relies on the ClassLoader.getResources method, that can be
overriden) but it doesn't look like (and probably intentionally so) to
be possible for named modules.

Guillaume


---
L'absence de virus dans ce courrier électronique a été vérifiée par le logiciel 
antivirus Avast.
https://www.avast.com/antivirus


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org




-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org




-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Loading providers in named modules with ServiceLoader using a plugin class realm

2017-06-25 Thread Chas Honton 
Under what circumstances would a plugin not want the platform classloader?

Chas

> On Jun 25, 2017, at 12:40 PM, Robert Scholte  wrote:
> 
> Hi Guillaume,
> 
> I don't know all the details about the Platform classloader, but it has been 
> introduced with a reason.
> So I don't think we should switch to it by default. I think the plugin is 
> well aware which classloaders / modules it wants to use (it should be), so I 
> think we need to find a mechanism for plugins to select their classloaders 
> and modules.
> 
> thanks,
> Robert
> 
>> On Sun, 25 Jun 2017 17:19:07 +0200, Guillaume Boué  wrote:
>> 
>> Hi,
>> 
>> With the introduction of modules in JDK 9, there were changes with
>> regard to how classloading works, and this impacts class realms created
>> in Maven. Today, the parent (as per ClassLoader.getParent()) of a class
>> realm is null, which represents the bootstrap classloader. In JDK 9, the
>> change is that some classes were moved to a named module other than
>> java.base, and they are not loaded with the bootstrap classloader
>> anymore, but with the platform classloader (which was previously the
>> extension classloader, see JDK-814637).
>> 
>> This has consequences, like MANTRUN-200, where locating providers with
>> the ServiceLoader API, using the plugin class realm, will miss JDK
>> internal implementation classes. In the case of MANTRUN-200, it is
>> Nashorn of the jdk.scripting.nashorn module, that cannot be found
>> because of the way ServiceLoader works
>> http://download.java.net/java/jdk9/docs/api/java/util/ServiceLoader.html#load-java.lang.Class-java.lang.ClassLoader-.
>> During the search in named modules, the class realm will not be able to
>> use its strategy since that process relies on parent delegation
>> implemented as explicit calls to ClassLoader.getParent()
>> (http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/f3cf7fd26baa/src/java.base/share/classes/java/util/ServiceLoader.java#l1062),
>> which, for a class realm, corresponds to the base classloader, i.e. the
>> bootstrap classloader. And since the Nashorn script engine factory was
>> loaded with the platform classloader, it is missed.
>> 
>> It seems that the fix here would be to make all class realms have as
>> base classloader the platform classloader starting with JDK 9, instead
>> of the bootstrap classloader (there is a new utility method in
>> ClassLoader to obtain the platform classloader). I verified that this
>> solves the problem described in MANTRUN-200, but before I create an MNG
>> issue, I'm wondering if this is the correct approach.
>> 
>> What do you think of this change to class realms? The other possibility
>> I can think of would be to have a way in the JDK to override the search
>> in named modules, so that our ClassRealm can also delegate to its parent
>> classloader. It is possible for unnamed modules (since the search
>> process then relies on the ClassLoader.getResources method, that can be
>> overriden) but it doesn't look like (and probably intentionally so) to
>> be possible for named modules.
>> 
>> Guillaume
>> 
>> 
>> ---
>> L'absence de virus dans ce courrier électronique a été vérifiée par le 
>> logiciel antivirus Avast.
>> https://www.avast.com/antivirus
>> 
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
>> For additional commands, e-mail: dev-h...@maven.apache.org
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
> 
> 


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Loading providers in named modules with ServiceLoader using a plugin class realm

2017-06-25 Thread Robert Scholte 

Hi Guillaume,

I don't know all the details about the Platform classloader, but it has  
been introduced with a reason.
So I don't think we should switch to it by default. I think the plugin is  
well aware which classloaders / modules it wants to use (it should be), so  
I think we need to find a mechanism for plugins to select their  
classloaders and modules.


thanks,
Robert

On Sun, 25 Jun 2017 17:19:07 +0200, Guillaume Boué   
wrote:



Hi,

With the introduction of modules in JDK 9, there were changes with
regard to how classloading works, and this impacts class realms created
in Maven. Today, the parent (as per ClassLoader.getParent()) of a class
realm is null, which represents the bootstrap classloader. In JDK 9, the
change is that some classes were moved to a named module other than
java.base, and they are not loaded with the bootstrap classloader
anymore, but with the platform classloader (which was previously the
extension classloader, see JDK-814637).

This has consequences, like MANTRUN-200, where locating providers with
the ServiceLoader API, using the plugin class realm, will miss JDK
internal implementation classes. In the case of MANTRUN-200, it is
Nashorn of the jdk.scripting.nashorn module, that cannot be found
because of the way ServiceLoader works
http://download.java.net/java/jdk9/docs/api/java/util/ServiceLoader.html#load-java.lang.Class-java.lang.ClassLoader-.
During the search in named modules, the class realm will not be able to
use its strategy since that process relies on parent delegation
implemented as explicit calls to ClassLoader.getParent()
(http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/f3cf7fd26baa/src/java.base/share/classes/java/util/ServiceLoader.java#l1062),
which, for a class realm, corresponds to the base classloader, i.e. the
bootstrap classloader. And since the Nashorn script engine factory was
loaded with the platform classloader, it is missed.

It seems that the fix here would be to make all class realms have as
base classloader the platform classloader starting with JDK 9, instead
of the bootstrap classloader (there is a new utility method in
ClassLoader to obtain the platform classloader). I verified that this
solves the problem described in MANTRUN-200, but before I create an MNG
issue, I'm wondering if this is the correct approach.

What do you think of this change to class realms? The other possibility
I can think of would be to have a way in the JDK to override the search
in named modules, so that our ClassRealm can also delegate to its parent
classloader. It is possible for unnamed modules (since the search
process then relies on the ClassLoader.getResources method, that can be
overriden) but it doesn't look like (and probably intentionally so) to
be possible for named modules.

Guillaume


---
L'absence de virus dans ce courrier électronique a été vérifiée par le  
logiciel antivirus Avast.

https://www.avast.com/antivirus


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Loading providers in named modules with ServiceLoader using a plugin class realm

2017-06-25 Thread Guillaume Boué 

Hi,

With the introduction of modules in JDK 9, there were changes with
regard to how classloading works, and this impacts class realms created
in Maven. Today, the parent (as per ClassLoader.getParent()) of a class
realm is null, which represents the bootstrap classloader. In JDK 9, the
change is that some classes were moved to a named module other than
java.base, and they are not loaded with the bootstrap classloader
anymore, but with the platform classloader (which was previously the
extension classloader, see JDK-814637).

This has consequences, like MANTRUN-200, where locating providers with
the ServiceLoader API, using the plugin class realm, will miss JDK
internal implementation classes. In the case of MANTRUN-200, it is
Nashorn of the jdk.scripting.nashorn module, that cannot be found
because of the way ServiceLoader works
http://download.java.net/java/jdk9/docs/api/java/util/ServiceLoader.html#load-java.lang.Class-java.lang.ClassLoader-.
During the search in named modules, the class realm will not be able to
use its strategy since that process relies on parent delegation
implemented as explicit calls to ClassLoader.getParent()
(http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/f3cf7fd26baa/src/java.base/share/classes/java/util/ServiceLoader.java#l1062),
which, for a class realm, corresponds to the base classloader, i.e. the
bootstrap classloader. And since the Nashorn script engine factory was
loaded with the platform classloader, it is missed.

It seems that the fix here would be to make all class realms have as
base classloader the platform classloader starting with JDK 9, instead
of the bootstrap classloader (there is a new utility method in
ClassLoader to obtain the platform classloader). I verified that this
solves the problem described in MANTRUN-200, but before I create an MNG
issue, I'm wondering if this is the correct approach.

What do you think of this change to class realms? The other possibility
I can think of would be to have a way in the JDK to override the search
in named modules, so that our ClassRealm can also delegate to its parent
classloader. It is possible for unnamed modules (since the search
process then relies on the ClassLoader.getResources method, that can be
overriden) but it doesn't look like (and probably intentionally so) to
be possible for named modules.

Guillaume


---
L'absence de virus dans ce courrier électronique a été vérifiée par le logiciel 
antivirus Avast.
https://www.avast.com/antivirus


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org