Re: Should ClassLoader::getResouces return the same resource twice?
Looks like the ModuleLayer.defineModules is the thing I needed. With it, I can use my current classloader to load all the modules and it can find the classes from both module and unnamed module. Note that class loaders for modules behave differently then original class loaders. For example they will favor local class definitions over class definitions of the parent loader. Not sure if there are other important differences. The loader is implemented in jdk.internal.loader.Loader [1]. There is some trick to get a class from an upper layer to bind to the a class in the boot layer? You can pass the created module layer as first argument to ServiceLoader.load(ModuleLayer, Class) [2]. [1] https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/jdk/internal/loader/Loader.java [2] https://docs.oracle.com/en/java/javase/15/docs/api/java.base/java/util/ServiceLoader.html#load(java.lang.ModuleLayer,java.lang.Class)
Re: Should ClassLoader::getResouces return the same resource twice?
The trick is that now I'm using my classloader to load the modules, I need to override the ClassLoader#findClass(String module, String className) So the service loader finds the classes successfully. Em seg., 4 de jan. de 2021 às 10:51, Thiago Henrique Hupner < thi...@gmail.com> escreveu: > Looks like the ModuleLayer.defineModules is the thing I needed. > With it, I can use my current classloader to load all the modules > and it can find the classes from both module and unnamed module. > > However, I'm having some trouble with the ServiceLoader now, > In the configuration, I've called resourceAndBind, but I'm getting > "Provider foo.bar not found" > > There is some trick to get a class from an upper layer to bind to the > a class in the boot layer? > > Thanks! > > Em seg., 4 de jan. de 2021 às 09:41, Johannes Spangenberg < > johannes.spangenb...@hotmail.de> escreveu: > >> The bidirectional delegation is an important point. Beside that, you >> also mentioned that your custom class loader stores additional >> information. >> >> > Unfortunately, I cannot only remove the jars from my classloader and >> > delegate >> > to the classloader provided by the ModuleLayer because we use a custom >> > classloader, >> > and have some important information stored in it, so by removing the >> > jars from the classloader >> > all applications would stop working. >> >> If you really need this information as part of the class loader, I fear >> you need to create custom class loaders for your modules together with >> ModuleLayer.defineModules [1] instead of >> ModuleLayer.defineModulesWithOneLoader. The class loader for >> ModuleLayer.defineModulesWithOneLoader is not public. I guess you would >> need to reimplement it. Maybe you can find other locations to store the >> information. >> >> [1] >> >> https://docs.oracle.com/en/java/javase/15/docs/api/java.base/java/lang/ModuleLayer.html#defineModules(java.lang.module.Configuration,java.util.List,java.util.function.Function) >> >> Am 04.01.2021 um 12:38 schrieb Thiago Henrique Hupner: >> > Yes, that's exactly it. Sorry for all the noise in the previous emails. >> > >> > I'm not using a URLClassLoader in my application, I've only used it >> > as an example because it is very easy to override to return a specific >> > value (and not have to implement all the other methods). >> > >> > So, I guess I will need to have a look at the application classloader >> > and update my current classloader to be able to do the bidirectional >> > delegation. Do you have any recommendations about where to look? >> > >> > Thanks! >> > >> > Em seg., 4 de jan. de 2021 às 05:48, Alan Bateman < >> alan.bate...@oracle.com> >> > escreveu: >> > >> >> On 03/01/2021 13:10, Thiago Henrique Hupner wrote: >> >>> : >> >>> >> >>> I hope made it a little more clear. By now it is clear that it is >> loading >> >>> the same >> >>> resource twice because the same jar is in the classloader and in the >> >> module >> >>> layer >> >>> classloader. >> >> If I read your mail correctly you've got a class path today and you >> want >> >> to "move" the well behaved components to a module path, leaving the >> >> remaining the components that can't work as automatic modules on the >> >> class path. It's equivalent to specifying both a module path and class >> >> path to the java command line launcher. You also mention that you have >> >> to use a custom class loader. If that class loader were updated to >> >> support modules and load classes into its unnamed module then you >> should >> >> be able to get this to work. This is very advanced territory and would >> >> be approximately equivalent to the application class loader in the JDK >> >> where it supports both modules and the class path. I don't think your >> >> current approach of using a URLClassLoader as a parent will work as it >> >> doesn't support the bidirectional delegation that would be need to get >> >> this to work. >> >> >> >> -Alan. >> >> >> >
Re: Should ClassLoader::getResouces return the same resource twice?
Looks like the ModuleLayer.defineModules is the thing I needed. With it, I can use my current classloader to load all the modules and it can find the classes from both module and unnamed module. However, I'm having some trouble with the ServiceLoader now, In the configuration, I've called resourceAndBind, but I'm getting "Provider foo.bar not found" There is some trick to get a class from an upper layer to bind to the a class in the boot layer? Thanks! Em seg., 4 de jan. de 2021 às 09:41, Johannes Spangenberg < johannes.spangenb...@hotmail.de> escreveu: > The bidirectional delegation is an important point. Beside that, you > also mentioned that your custom class loader stores additional information. > > > Unfortunately, I cannot only remove the jars from my classloader and > > delegate > > to the classloader provided by the ModuleLayer because we use a custom > > classloader, > > and have some important information stored in it, so by removing the > > jars from the classloader > > all applications would stop working. > > If you really need this information as part of the class loader, I fear > you need to create custom class loaders for your modules together with > ModuleLayer.defineModules [1] instead of > ModuleLayer.defineModulesWithOneLoader. The class loader for > ModuleLayer.defineModulesWithOneLoader is not public. I guess you would > need to reimplement it. Maybe you can find other locations to store the > information. > > [1] > > https://docs.oracle.com/en/java/javase/15/docs/api/java.base/java/lang/ModuleLayer.html#defineModules(java.lang.module.Configuration,java.util.List,java.util.function.Function) > > Am 04.01.2021 um 12:38 schrieb Thiago Henrique Hupner: > > Yes, that's exactly it. Sorry for all the noise in the previous emails. > > > > I'm not using a URLClassLoader in my application, I've only used it > > as an example because it is very easy to override to return a specific > > value (and not have to implement all the other methods). > > > > So, I guess I will need to have a look at the application classloader > > and update my current classloader to be able to do the bidirectional > > delegation. Do you have any recommendations about where to look? > > > > Thanks! > > > > Em seg., 4 de jan. de 2021 às 05:48, Alan Bateman < > alan.bate...@oracle.com> > > escreveu: > > > >> On 03/01/2021 13:10, Thiago Henrique Hupner wrote: > >>> : > >>> > >>> I hope made it a little more clear. By now it is clear that it is > loading > >>> the same > >>> resource twice because the same jar is in the classloader and in the > >> module > >>> layer > >>> classloader. > >> If I read your mail correctly you've got a class path today and you want > >> to "move" the well behaved components to a module path, leaving the > >> remaining the components that can't work as automatic modules on the > >> class path. It's equivalent to specifying both a module path and class > >> path to the java command line launcher. You also mention that you have > >> to use a custom class loader. If that class loader were updated to > >> support modules and load classes into its unnamed module then you should > >> be able to get this to work. This is very advanced territory and would > >> be approximately equivalent to the application class loader in the JDK > >> where it supports both modules and the class path. I don't think your > >> current approach of using a URLClassLoader as a parent will work as it > >> doesn't support the bidirectional delegation that would be need to get > >> this to work. > >> > >> -Alan. > >> >
Re: Should ClassLoader::getResouces return the same resource twice?
The bidirectional delegation is an important point. Beside that, you also mentioned that your custom class loader stores additional information. Unfortunately, I cannot only remove the jars from my classloader and delegate to the classloader provided by the ModuleLayer because we use a custom classloader, and have some important information stored in it, so by removing the jars from the classloader all applications would stop working. If you really need this information as part of the class loader, I fear you need to create custom class loaders for your modules together with ModuleLayer.defineModules [1] instead of ModuleLayer.defineModulesWithOneLoader. The class loader for ModuleLayer.defineModulesWithOneLoader is not public. I guess you would need to reimplement it. Maybe you can find other locations to store the information. [1] https://docs.oracle.com/en/java/javase/15/docs/api/java.base/java/lang/ModuleLayer.html#defineModules(java.lang.module.Configuration,java.util.List,java.util.function.Function) Am 04.01.2021 um 12:38 schrieb Thiago Henrique Hupner: Yes, that's exactly it. Sorry for all the noise in the previous emails. I'm not using a URLClassLoader in my application, I've only used it as an example because it is very easy to override to return a specific value (and not have to implement all the other methods). So, I guess I will need to have a look at the application classloader and update my current classloader to be able to do the bidirectional delegation. Do you have any recommendations about where to look? Thanks! Em seg., 4 de jan. de 2021 às 05:48, Alan Bateman escreveu: On 03/01/2021 13:10, Thiago Henrique Hupner wrote: : I hope made it a little more clear. By now it is clear that it is loading the same resource twice because the same jar is in the classloader and in the module layer classloader. If I read your mail correctly you've got a class path today and you want to "move" the well behaved components to a module path, leaving the remaining the components that can't work as automatic modules on the class path. It's equivalent to specifying both a module path and class path to the java command line launcher. You also mention that you have to use a custom class loader. If that class loader were updated to support modules and load classes into its unnamed module then you should be able to get this to work. This is very advanced territory and would be approximately equivalent to the application class loader in the JDK where it supports both modules and the class path. I don't think your current approach of using a URLClassLoader as a parent will work as it doesn't support the bidirectional delegation that would be need to get this to work. -Alan.
Re: Should ClassLoader::getResouces return the same resource twice?
Yes, that's exactly it. Sorry for all the noise in the previous emails. I'm not using a URLClassLoader in my application, I've only used it as an example because it is very easy to override to return a specific value (and not have to implement all the other methods). So, I guess I will need to have a look at the application classloader and update my current classloader to be able to do the bidirectional delegation. Do you have any recommendations about where to look? Thanks! Em seg., 4 de jan. de 2021 às 05:48, Alan Bateman escreveu: > On 03/01/2021 13:10, Thiago Henrique Hupner wrote: > > : > > > > I hope made it a little more clear. By now it is clear that it is loading > > the same > > resource twice because the same jar is in the classloader and in the > module > > layer > > classloader. > If I read your mail correctly you've got a class path today and you want > to "move" the well behaved components to a module path, leaving the > remaining the components that can't work as automatic modules on the > class path. It's equivalent to specifying both a module path and class > path to the java command line launcher. You also mention that you have > to use a custom class loader. If that class loader were updated to > support modules and load classes into its unnamed module then you should > be able to get this to work. This is very advanced territory and would > be approximately equivalent to the application class loader in the JDK > where it supports both modules and the class path. I don't think your > current approach of using a URLClassLoader as a parent will work as it > doesn't support the bidirectional delegation that would be need to get > this to work. > > -Alan. >
Re: Should ClassLoader::getResouces return the same resource twice?
On 03/01/2021 13:10, Thiago Henrique Hupner wrote: : I hope made it a little more clear. By now it is clear that it is loading the same resource twice because the same jar is in the classloader and in the module layer classloader. If I read your mail correctly you've got a class path today and you want to "move" the well behaved components to a module path, leaving the remaining the components that can't work as automatic modules on the class path. It's equivalent to specifying both a module path and class path to the java command line launcher. You also mention that you have to use a custom class loader. If that class loader were updated to support modules and load classes into its unnamed module then you should be able to get this to work. This is very advanced territory and would be approximately equivalent to the application class loader in the JDK where it supports both modules and the class path. I don't think your current approach of using a URLClassLoader as a parent will work as it doesn't support the bidirectional delegation that would be need to get this to work. -Alan.
Re: Should ClassLoader::getResouces return the same resource twice?
Yes, that is intentional. So I guess the correct behavior is to find the jars only via the ModuleFinder, and use the classloader only for nonmodular jars, is this right? In the Servlet spec, it is allowed to have split packages, from the WEB-INF/classes and WEB-INF/lib/ jars, for instance, so it is not as trivial as running an application in the command line. Besides being allowed split packages, the classes in the WEB-INF/classes must override the same classes in the WEB-INF/lib (something like the --patch-module) My current behavior is trying to find the modules using my ModuleFinder (most of the modules will be an automatic module) and if it is not allowed for some reason (invalid package name, invalid services, so on) it fallbacks to the classloader to still find all the things correctly. Unfortunately, I cannot only remove the jars from my classloader and delegate to the classloader provided by the ModuleLayer because we use a custom classloader, and have some important information stored in it, so by removing the jars from the classloader all applications would stop working. I hope made it a little more clear. By now it is clear that it is loading the same resource twice because the same jar is in the classloader and in the module layer classloader. Em dom., 3 de jan. de 2021 às 01:24, Johannes Spangenberg < johannes.spangenb...@hotmail.de> escreveu: > Note that you are actually loading the JARs twice which means that you > might have two versions of the same classses loaded, not only of the > resources. Is this intentional? I still don't know what you want to > achieve with your class loader and module layer. > > Am 02.01.2021 um 21:06 schrieb Thiago Henrique Hupner: > > Yes, the parent loader locates two resources and it is used in the > > defineModulesWithOneLoader. > > > > So the behavior is the following: > > module-a.jar -> META-INF/web-fragment.xml > > module-b.jar -> META-INF/web-fragment.xml > > > > Creating a URLClassloader with these two jars and calling the > getResources, > > it returns two URLs. > > > > If then I create a module for each jar, the contents of each module will > be > > the same as the content > > found by the URLClassloader. > > Using that classloader as the parent in the defineModulesWithOneLoader > > gives the following behavior: > > > > Calling getResources with the loader created by the ModuleLayer, > > it will search the resources in all the modules of that layer (module.a > and > > module.b), > > and then delegate to the parent (URLClassloader), which will find the > same > > resources, > > thus, doubling the results. > > > > Now describing it make clear to me that is something not easy to fix, > > so by now, I worked around it by creating a ModuleReader that only > > find the ".class" files in the module and the other resources will be > > available by the parent, > > however, I don't know if this solution is the best. > > > > > > Em sáb., 2 de jan. de 2021 às 14:51, Alan Bateman < > alan.bate...@oracle.com> > > escreveu: > > > >> On 02/01/2021 12:59, Thiago Henrique Hupner wrote: > >>> I guess a little context can make more things clear: > >>> The servlet spec requires that all jars from WEB-INF/lib > >>> be available to the same classloader. > >>> The resource, in particular, is "META-INF/web-fragment.xml" > >>> Each jar can contain its own. So, using getResources make sense > >>> in order of parsing each. However, what is happening is if I have two > >> JARs > >>> each with its own META-INF/web-fragment.xml, using the ModuleReader > >>> it is returning four resources, so it parses more than it should and > >>> it fails > >>> to parse the same resource twice. > >>> > >> Which parent class loader are you specifying to > >> defineModulesWithOneLoader? It it possible that it locates the resource > >> in the two JAR files? > >> > >> -Alan > >> >
Re: Should ClassLoader::getResouces return the same resource twice?
Note that you are actually loading the JARs twice which means that you might have two versions of the same classses loaded, not only of the resources. Is this intentional? I still don't know what you want to achieve with your class loader and module layer. Am 02.01.2021 um 21:06 schrieb Thiago Henrique Hupner: Yes, the parent loader locates two resources and it is used in the defineModulesWithOneLoader. So the behavior is the following: module-a.jar -> META-INF/web-fragment.xml module-b.jar -> META-INF/web-fragment.xml Creating a URLClassloader with these two jars and calling the getResources, it returns two URLs. If then I create a module for each jar, the contents of each module will be the same as the content found by the URLClassloader. Using that classloader as the parent in the defineModulesWithOneLoader gives the following behavior: Calling getResources with the loader created by the ModuleLayer, it will search the resources in all the modules of that layer (module.a and module.b), and then delegate to the parent (URLClassloader), which will find the same resources, thus, doubling the results. Now describing it make clear to me that is something not easy to fix, so by now, I worked around it by creating a ModuleReader that only find the ".class" files in the module and the other resources will be available by the parent, however, I don't know if this solution is the best. Em sáb., 2 de jan. de 2021 às 14:51, Alan Bateman escreveu: On 02/01/2021 12:59, Thiago Henrique Hupner wrote: I guess a little context can make more things clear: The servlet spec requires that all jars from WEB-INF/lib be available to the same classloader. The resource, in particular, is "META-INF/web-fragment.xml" Each jar can contain its own. So, using getResources make sense in order of parsing each. However, what is happening is if I have two JARs each with its own META-INF/web-fragment.xml, using the ModuleReader it is returning four resources, so it parses more than it should and it fails to parse the same resource twice. Which parent class loader are you specifying to defineModulesWithOneLoader? It it possible that it locates the resource in the two JAR files? -Alan
Re: Should ClassLoader::getResouces return the same resource twice?
Yes, the parent loader locates two resources and it is used in the defineModulesWithOneLoader. So the behavior is the following: module-a.jar -> META-INF/web-fragment.xml module-b.jar -> META-INF/web-fragment.xml Creating a URLClassloader with these two jars and calling the getResources, it returns two URLs. If then I create a module for each jar, the contents of each module will be the same as the content found by the URLClassloader. Using that classloader as the parent in the defineModulesWithOneLoader gives the following behavior: Calling getResources with the loader created by the ModuleLayer, it will search the resources in all the modules of that layer (module.a and module.b), and then delegate to the parent (URLClassloader), which will find the same resources, thus, doubling the results. Now describing it make clear to me that is something not easy to fix, so by now, I worked around it by creating a ModuleReader that only find the ".class" files in the module and the other resources will be available by the parent, however, I don't know if this solution is the best. Em sáb., 2 de jan. de 2021 às 14:51, Alan Bateman escreveu: > On 02/01/2021 12:59, Thiago Henrique Hupner wrote: > > I guess a little context can make more things clear: > > The servlet spec requires that all jars from WEB-INF/lib > > be available to the same classloader. > > The resource, in particular, is "META-INF/web-fragment.xml" > > Each jar can contain its own. So, using getResources make sense > > in order of parsing each. However, what is happening is if I have two > JARs > > each with its own META-INF/web-fragment.xml, using the ModuleReader > > it is returning four resources, so it parses more than it should and > > it fails > > to parse the same resource twice. > > > Which parent class loader are you specifying to > defineModulesWithOneLoader? It it possible that it locates the resource > in the two JAR files? > > -Alan >
Re: Should ClassLoader::getResouces return the same resource twice?
On 02/01/2021 12:59, Thiago Henrique Hupner wrote: I guess a little context can make more things clear: The servlet spec requires that all jars from WEB-INF/lib be available to the same classloader. The resource, in particular, is "META-INF/web-fragment.xml" Each jar can contain its own. So, using getResources make sense in order of parsing each. However, what is happening is if I have two JARs each with its own META-INF/web-fragment.xml, using the ModuleReader it is returning four resources, so it parses more than it should and it fails to parse the same resource twice. Which parent class loader are you specifying to defineModulesWithOneLoader? It it possible that it locates the resource in the two JAR files? -Alan
Re: Should ClassLoader::getResouces return the same resource twice?
As an aside, creating a FileSystem is not an onerous task. https://stackoverflow.com/questions/22966176/creating-a-custom-filesystem-implementation-in-java Cheers, — Jim > On Jan 2, 2021, at 8:59 AM, Thiago Henrique Hupner wrote: > > I guess a little context can make more things clear: > The servlet spec requires that all jars from WEB-INF/lib > be available to the same classloader. > The resource, in particular, is "META-INF/web-fragment.xml" > Each jar can contain its own. So, using getResources make sense > in order of parsing each. However, what is happening is if I have two JARs > each with its own META-INF/web-fragment.xml, using the ModuleReader > it is returning four resources, so it parses more than it should and it > fails > to parse the same resource twice. > > I'll have a try only exposing the ".class" files in the ModuleReader, > so the Loader will be able to create the classes and it will read the > resources > from the classloader. > > I'm using my own implementations of the ModuleReader, ModuleFinder > and ModuleDescriptor because all of our resources are in memory I wasn't > able to > use the ModuleFinder.of() because it requires a filesystem. > >> Em sáb., 2 de jan. de 2021 às 04:07, Alan Bateman >> escreveu: >> >>> On 02/01/2021 03:21, Thiago Henrique Hupner wrote: >>> : >>> >>> I've created a simple example of what is occurring [1]. >>> I know there are behavior specific for getting a class if it is in a >> module, >>> but I don't know if this may be a bug in the resource loading mechanism. >>> In the example, the returned values are different to illustrate, but in >> my >>> case, it >>> returns two exact URLs for the same resource as the source for the module >>> reader >>> and the classloader is the same. >> The behavior you observe with the example is correct. >> ClassLoader.getResources locates the resource by searching the lass >> loader delegation chain and in the example there are two "foo" >> resources. The resource in module fake.module is located because it is >> an automatic module that opens all its packages unconditionally. The >> second resource is located by searching the parent class loader, a >> URLClassLoader in the example that also locates "foo". Which "foo" did >> you expect to locate? If code in fake.module just wants to locate the >> resource in its own module then it should use getResource rather than >> getResources. >> >> -Alan. >>
Re: Should ClassLoader::getResouces return the same resource twice?
I guess a little context can make more things clear: The servlet spec requires that all jars from WEB-INF/lib be available to the same classloader. The resource, in particular, is "META-INF/web-fragment.xml" Each jar can contain its own. So, using getResources make sense in order of parsing each. However, what is happening is if I have two JARs each with its own META-INF/web-fragment.xml, using the ModuleReader it is returning four resources, so it parses more than it should and it fails to parse the same resource twice. I'll have a try only exposing the ".class" files in the ModuleReader, so the Loader will be able to create the classes and it will read the resources from the classloader. I'm using my own implementations of the ModuleReader, ModuleFinder and ModuleDescriptor because all of our resources are in memory I wasn't able to use the ModuleFinder.of() because it requires a filesystem. Em sáb., 2 de jan. de 2021 às 04:07, Alan Bateman escreveu: > On 02/01/2021 03:21, Thiago Henrique Hupner wrote: > > : > > > > I've created a simple example of what is occurring [1]. > > I know there are behavior specific for getting a class if it is in a > module, > > but I don't know if this may be a bug in the resource loading mechanism. > > In the example, the returned values are different to illustrate, but in > my > > case, it > > returns two exact URLs for the same resource as the source for the module > > reader > > and the classloader is the same. > The behavior you observe with the example is correct. > ClassLoader.getResources locates the resource by searching the lass > loader delegation chain and in the example there are two "foo" > resources. The resource in module fake.module is located because it is > an automatic module that opens all its packages unconditionally. The > second resource is located by searching the parent class loader, a > URLClassLoader in the example that also locates "foo". Which "foo" did > you expect to locate? If code in fake.module just wants to locate the > resource in its own module then it should use getResource rather than > getResources. > > -Alan. >
Re: Should ClassLoader::getResouces return the same resource twice?
On 02/01/2021 03:21, Thiago Henrique Hupner wrote: : I've created a simple example of what is occurring [1]. I know there are behavior specific for getting a class if it is in a module, but I don't know if this may be a bug in the resource loading mechanism. In the example, the returned values are different to illustrate, but in my case, it returns two exact URLs for the same resource as the source for the module reader and the classloader is the same. The behavior you observe with the example is correct. ClassLoader.getResources locates the resource by searching the lass loader delegation chain and in the example there are two "foo" resources. The resource in module fake.module is located because it is an automatic module that opens all its packages unconditionally. The second resource is located by searching the parent class loader, a URLClassLoader in the example that also locates "foo". Which "foo" did you expect to locate? If code in fake.module just wants to locate the resource in its own module then it should use getResource rather than getResources. -Alan.
Should ClassLoader::getResouces return the same resource twice?
Hi! I'm working to integrate the JPMS into our application server in a way that the applications can run in a layer each, but I faced something strange: our current behavior without layers, calling ClassLoader::getResources in the classloader provided by us, it will return a return "foo". In the conversion to layer, it was clear to me that I would need to use the ModuleLayer::defineModulesWithOneLoader, so the application classloading behavior wouldn't change and the ModuleReader would use the same source of resources that the classloader is using. However, calling the ClassLoader::getResources in the classloader provided by the Module system, it is finding the resource in itself and on the parent, so it finds the same resource twice. As our applications aren't expected to work with the same resource twice, it broke in some places where we cannot change. I've created a simple example of what is occurring [1]. I know there are behavior specific for getting a class if it is in a module, but I don't know if this may be a bug in the resource loading mechanism. In the example, the returned values are different to illustrate, but in my case, it returns two exact URLs for the same resource as the source for the module reader and the classloader is the same. [1] https://gist.github.com/Thihup/8e53bc991272452e63a6055f05690ba5
