Re: ClassLoader.getResources(String)

[Alan Bateman]

On 07/03/2018 15:14, Stephen Colebourne wrote:

:
Configuration and code are two very different things. Asking projects
and end users to write code for something that should be config is a
huge no-no.

My view is that JPMS has made using configuration files, especially
for libraries, a lot harder. This is a step back in usability. Just so
we are clear, for leap seconds I will now have to ask users to
manually register them using an API where previously they just added a
file. But for OpenGamma Strata, the configuration files are much more
complex and certainly unsuited to be code, even if the backwards
compatibility issues were acceptable. (This is a pattern I've used for
configuration for many years)
It's usually cleaner to encapsulate that configuration but if you don't 
want to change anything then you can continue to use 
ClassLoader.getResources to search for resources as it works exactly as 
it did before. Also if you move the configuration file to somewhere like 
META-INF/config then it can never be encapsulated.



:


PS. ServiceLoader is a pain to use in Java 9 too. As a library doesn't
know whether it will run as a named module or on the classpath, I have
to duplicate the service loader configuration - once in
META-INF/services and once in module-info.java, which is horrible.

Tooling should be able help with cases where you are creating a library 
that may be deployed on the class path in some environments and the 
module path in others. The `jar` tool does some sanity check in this 
area but it could do more.


-Alan

Re: ClassLoader.getResources(String)

[Mark Raynsford]

On 2018-03-07T15:14:11 +
Stephen Colebourne  wrote:
>
> Effectively what is needed is another way for a library to be informed
> of the presence of the calling application. One possible solution to
> this would be to allow users to write module initialization code in
> module-info.java. Then an application coder would have a solid
> reliable place to put code that registers the additional configuration
> files with the low-level library.

There's a pattern I've seen used in OSGi that I've considered adapting
for my own use (exposing resource-only modules that don't contain any
code and yet need to be able to cause some other piece of code to
instantiate services on behalf of the module): They call it the
"extender pattern".

  https://dzone.com/articles/osgi-42-extender-pattern-and

Briefly, what happens is that you subscribe to an interface that tells
you when bundles (OSGi terminology for artifacts containing modules,
more or less) are added to or removed from the system. In Jigsaw, this
would probably equate to publishing some sort of event that can be
observed whenever someone creates a new module layer. When you get
notified that a bundle has been added, you can scan the manifest of the
bundle (via the standard jar manifest APIs if you like) and can then,
for example, look for manifest fields that tell you where in the jar
file to find application-specific config files.

  X-My-Extra-Config-File: /com/example/config.xml

The listening party can then read the config file, instantiate services
as necessary, etc.

It'd need to be handled in a way that ensured that you don't
essentially race the module resolution code; Just because your code
didn't get a chance to subscribe to "module became available" events
until after they'd appeared doesn't mean you should miss the events.
OSGi does handle this (it's written into the spec), but I don't know
quite how the implementations handle it.

-- 
Mark Raynsford | http://www.io7m.com

Re: ClassLoader.getResources(String)

[Stephen Colebourne]

On 7 March 2018 at 12:59, Alan Bateman  wrote:
> You've dismissed services but I would expect it to provide a nice solution.
> The service interface might be very simple, something like:
>
> public interface LeapSecondDataProvider {
> LeapSecondData data();
> }

Configuration and code are two very different things. Asking projects
and end users to write code for something that should be config is a
huge no-no.

My view is that JPMS has made using configuration files, especially
for libraries, a lot harder. This is a step back in usability. Just so
we are clear, for leap seconds I will now have to ask users to
manually register them using an API where previously they just added a
file. But for OpenGamma Strata, the configuration files are much more
complex and certainly unsuited to be code, even if the backwards
compatibility issues were acceptable. (This is a pattern I've used for
configuration for many years)

Effectively what is needed is another way for a library to be informed
of the presence of the calling application. One possible solution to
this would be to allow users to write module initialization code in
module-info.java. Then an application coder would have a solid
reliable place to put code that registers the additional configuration
files with the low-level library. Something like:

module com.foo.app {
  requires org.threeten.extra;

  init(ModuleInitContext context) {
UtcRules.registerLeapSecondFile("/com/foo/app/LeapSeconds.txt");
  }
}

PS. ServiceLoader is a pain to use in Java 9 too. As a library doesn't
know whether it will run as a named module or on the classpath, I have
to duplicate the service loader configuration - once in
META-INF/services and once in module-info.java, which is horrible. It
also means that the provide() static method is a feature that cannot
be used by libraries.

Stephen

Re: ClassLoader.getResources(String)

[Alan Bateman]

On 07/03/2018 12:11, Stephen Colebourne wrote:

Following up on this, it does feel like the use case is now simply not
possible. I have a similar problem with
ClassLoader.getResources(String) in threeten-extra.

https://github.com/ThreeTen/threetenbp-extra/blob/master/src/main/java/org/threeten/extra/scale/SystemUTCRules.java#L202

The ThreeTen-Extra project defines a config file
org/threeten/extra/scale/LeapSecond.txt. The code uses
ClassLoader.getResources(String) to find the latest version of the
file, which may be in the threeten-extra jar file, or in any jar file
that uses threeten-extra.jar. ie. to replace the version from
threeten-extra.jar, a user simply has to add a file with the same
name/package to their jar file.

threeten-extra.jar contains org/threeten/extra/scale/LeapSecond.txt
application.jar also contains org/threeten/extra/scale/LeapSecond.txt

Under JPMS this fails, as the resource cannot be located in
org/threeten/extra/scale in a different jar file. But this appears
makes the whole design impossible to make work with JPMS.

The code in threeten-extra.jar cannot possibly know about the package
names of the application.jar, so there is no way for it to find the
config file.

There seem to be only two solutions to this
- ServiceLoader, but that is for code, not config files
- forcing the application to manually register their config file

Both of these provide a markedly worse outcome.

Am I missing something?

Resources can't be both encapsulated and not encapsulated at the same 
time. If a module has a resource in a package that is intended to be 
located by code in other modules using ClassLoader getResourceXXX then 
it has to open the package.


In the above, then I assume the main issue isn't resource encapsulation, 
it's that you've got two modules on the application module class 
containing the same package so they can't both be mapped to the 
application class loader.


You've dismissed services but I would expect it to provide a nice 
solution. The service interface might be very simple, something like:


public interface LeapSecondDataProvider {
    LeapSecondData data();
}

or better still, define methods that allow SystemUTCRules select the 
right version of the leap second data. Applications that ship their own 
leap data second would ship an implementation of this class. Yes, it's 
different to searching the class file for configuration files but a lot 
more reliable.


-Alan

Re: ClassLoader.getResources(String)

[Stephen Colebourne]

Following up on this, it does feel like the use case is now simply not
possible. I have a similar problem with
ClassLoader.getResources(String) in threeten-extra.

https://github.com/ThreeTen/threetenbp-extra/blob/master/src/main/java/org/threeten/extra/scale/SystemUTCRules.java#L202

The ThreeTen-Extra project defines a config file
org/threeten/extra/scale/LeapSecond.txt. The code uses
ClassLoader.getResources(String) to find the latest version of the
file, which may be in the threeten-extra jar file, or in any jar file
that uses threeten-extra.jar. ie. to replace the version from
threeten-extra.jar, a user simply has to add a file with the same
name/package to their jar file.

threeten-extra.jar contains org/threeten/extra/scale/LeapSecond.txt
application.jar also contains org/threeten/extra/scale/LeapSecond.txt

Under JPMS this fails, as the resource cannot be located in
org/threeten/extra/scale in a different jar file. But this appears
makes the whole design impossible to make work with JPMS.

The code in threeten-extra.jar cannot possibly know about the package
names of the application.jar, so there is no way for it to find the
config file.

There seem to be only two solutions to this
- ServiceLoader, but that is for code, not config files
- forcing the application to manually register their config file

Both of these provide a markedly worse outcome.

Am I missing something?

Stephen


On 7 February 2018 at 20:11, Alan Bateman <alan.bate...@oracle.com> wrote:
> On 07/02/2018 16:56, Stephen Colebourne wrote:
>>
>> :
>> I was using maven to create a jar-with-dependencies file, so I could
>> use jlink. With all the code in one jar file, there shouldn't be any
>> access barriers to worry about.
>>
>> ClassLoader.getResources(String) worked just fine until Java 9. The
>> two APIs are not comparable - the ClassLoader one returns all URLs
>> found, whereas the Class one returns just one URL. Switching API would
>> change behaviour.
>
> ClassLoader.getResources searches the class path as it did in JDK 9 and
> older, it it just can't locate non-".class" resources in modules when they
> are encapsulated. Class loaders are oblivious as to who is ultimately
> attempting to load a class or locate a resource (the initiating and defining
> loader can be different, they can many class loaders in the delegation
> chain).
>
> With the uber modular JAR scenario then all classes for several libraries
> are in the same module. This means that the names of resources in that
> module are unique. If several libraries have the same resource then I assume
> you drop all but one when you create this uber JAR (or maybe you are merging
> some of the configuration files, I can't tell). So I assume you could change
> this code to use Class.getResource and it will locate at-most-one resource
> with a specific name.
>
> To do a proper migration means re-examining ResourceConfig of course. Using
> services is likely to be a lot cleaner and more robust than scanning for
> configuration files.
>
> -Alan

Re: ClassLoader.getResources(String)

[Alan Bateman]

On 07/02/2018 16:56, Stephen Colebourne wrote:

:
I was using maven to create a jar-with-dependencies file, so I could
use jlink. With all the code in one jar file, there shouldn't be any
access barriers to worry about.

ClassLoader.getResources(String) worked just fine until Java 9. The
two APIs are not comparable - the ClassLoader one returns all URLs
found, whereas the Class one returns just one URL. Switching API would
change behaviour.
ClassLoader.getResources searches the class path as it did in JDK 9 and 
older, it it just can't locate non-".class" resources in modules when 
they are encapsulated. Class loaders are oblivious as to who is 
ultimately attempting to load a class or locate a resource (the 
initiating and defining loader can be different, they can many class 
loaders in the delegation chain).


With the uber modular JAR scenario then all classes for several 
libraries are in the same module. This means that the names of resources 
in that module are unique. If several libraries have the same resource 
then I assume you drop all but one when you create this uber JAR (or 
maybe you are merging some of the configuration files, I can't tell). So 
I assume you could change this code to use Class.getResource and it will 
locate at-most-one resource with a specific name.


To do a proper migration means re-examining ResourceConfig of course. 
Using services is likely to be a lot cleaner and more robust than 
scanning for configuration files.


-Alan

Re: ClassLoader.getResources(String)

[Stephen Colebourne]

On 7 February 2018 at 16:35, Alan Bateman <alan.bate...@oracle.com> wrote:
> On 07/02/2018 14:23, Stephen Colebourne wrote:
>>
>> I've been trying to use ClassLoader.getResources(String). The entire
>> application is in one named module, this includes the code that
>> invokes the ClassLoader method and the resource that it is trying to
>> find.
>
> Can you summarize what you are trying to do? If this is code in a module
> trying to locate one of its own resources then Class.getResourceXXX or
> Module.getResourcAsStream are the candidate APIs to use (not
> ClassLoader.getResourceXXX as that can never locate resources that are
> encapsulated).

I was using maven to create a jar-with-dependencies file, so I could
use jlink. With all the code in one jar file, there shouldn't be any
access barriers to worry about.

ClassLoader.getResources(String) worked just fine until Java 9. The
two APIs are not comparable - the ClassLoader one returns all URLs
found, whereas the Class one returns just one URL. Switching API would
change behaviour.

The code can be seen here:
https://github.com/OpenGamma/Strata/blob/master/modules/collect/src/main/java/com/opengamma/strata/collect/io/ResourceConfig.java#L242
It is a core part of the system that loads configuration at startup.

thanks
Stephen

Re: ClassLoader.getResources(String)

[Alan Bateman]

On 07/02/2018 14:23, Stephen Colebourne wrote:

I've been trying to use ClassLoader.getResources(String). The entire
application is in one named module, this includes the code that
invokes the ClassLoader method and the resource that it is trying to
find.
Can you summarize what you are trying to do? If this is code in a module 
trying to locate one of its own resources then Class.getResourceXXX or 
Module.getResourcAsStream are the candidate APIs to use (not 
ClassLoader.getResourceXXX as that can never locate resources that are 
encapsulated).


-Alan

ClassLoader.getResources(String)

[Stephen Colebourne]

I've been trying to use ClassLoader.getResources(String). The entire
application is in one named module, this includes the code that
invokes the ClassLoader method and the resource that it is trying to
find. The Javadoc says:

"Resources in named modules are subject to the encapsulation rules
specified by Module.getResourceAsStream. Additionally, and except for
the special case where the resource has a name ending with ".class",
this method will only find resources in packages of named modules when
the package is opened unconditionally (even if the caller of this
method is in the same module as the resource)."
https://docs.oracle.com/javase/9/docs/api/java/lang/ClassLoader.html#getResources-java.lang.String-

The call to ClassLoader.getResources(String) does not find the
resource. I assume that this is because of the last clause in the spec
"even if the caller of this method is in the same module as the
resource".

But I can't for the life of me think why such a difficult to meet
restriction has been added. The only way around it is to make the
package open, which is far from ideal. If its all within one module,
applying an access restriction like this is just unhelpful.

Stephen