Re: ClassLoader.getResources(String)

[Alan Bateman]

On 07/03/2018 15:14, Stephen Colebourne wrote:

:
Configuration and code are two very different things. Asking projects
and end users to write code for something that should be config is a
huge no-no.

My view is that JPMS has made using configuration files, especially
for libraries, a lot harder. This is a step back in usability. Just so
we are clear, for leap seconds I will now have to ask users to
manually register them using an API where previously they just added a
file. But for OpenGamma Strata, the configuration files are much more
complex and certainly unsuited to be code, even if the backwards
compatibility issues were acceptable. (This is a pattern I've used for
configuration for many years)
It's usually cleaner to encapsulate that configuration but if you don't 
want to change anything then you can continue to use 
ClassLoader.getResources to search for resources as it works exactly as 
it did before. Also if you move the configuration file to somewhere like 
META-INF/config then it can never be encapsulated.



:


PS. ServiceLoader is a pain to use in Java 9 too. As a library doesn't
know whether it will run as a named module or on the classpath, I have
to duplicate the service loader configuration - once in
META-INF/services and once in module-info.java, which is horrible.

Tooling should be able help with cases where you are creating a library 
that may be deployed on the class path in some environments and the 
module path in others. The `jar` tool does some sanity check in this 
area but it could do more.


-Alan

Re: ClassLoader.getResources(String)

[Mark Raynsford]

On 2018-03-07T15:14:11 +
Stephen Colebourne  wrote:
>
> Effectively what is needed is another way for a library to be informed
> of the presence of the calling application. One possible solution to
> this would be to allow users to write module initialization code in
> module-info.java. Then an application coder would have a solid
> reliable place to put code that registers the additional configuration
> files with the low-level library.

There's a pattern I've seen used in OSGi that I've considered adapting
for my own use (exposing resource-only modules that don't contain any
code and yet need to be able to cause some other piece of code to
instantiate services on behalf of the module): They call it the
"extender pattern".

  https://dzone.com/articles/osgi-42-extender-pattern-and

Briefly, what happens is that you subscribe to an interface that tells
you when bundles (OSGi terminology for artifacts containing modules,
more or less) are added to or removed from the system. In Jigsaw, this
would probably equate to publishing some sort of event that can be
observed whenever someone creates a new module layer. When you get
notified that a bundle has been added, you can scan the manifest of the
bundle (via the standard jar manifest APIs if you like) and can then,
for example, look for manifest fields that tell you where in the jar
file to find application-specific config files.

  X-My-Extra-Config-File: /com/example/config.xml

The listening party can then read the config file, instantiate services
as necessary, etc.

It'd need to be handled in a way that ensured that you don't
essentially race the module resolution code; Just because your code
didn't get a chance to subscribe to "module became available" events
until after they'd appeared doesn't mean you should miss the events.
OSGi does handle this (it's written into the spec), but I don't know
quite how the implementations handle it.

-- 
Mark Raynsford | http://www.io7m.com

Re: ClassLoader.getResources(String)

[Stephen Colebourne]

On 7 March 2018 at 12:59, Alan Bateman  wrote:
> You've dismissed services but I would expect it to provide a nice solution.
> The service interface might be very simple, something like:
>
> public interface LeapSecondDataProvider {
> LeapSecondData data();
> }

Configuration and code are two very different things. Asking projects
and end users to write code for something that should be config is a
huge no-no.

My view is that JPMS has made using configuration files, especially
for libraries, a lot harder. This is a step back in usability. Just so
we are clear, for leap seconds I will now have to ask users to
manually register them using an API where previously they just added a
file. But for OpenGamma Strata, the configuration files are much more
complex and certainly unsuited to be code, even if the backwards
compatibility issues were acceptable. (This is a pattern I've used for
configuration for many years)

Effectively what is needed is another way for a library to be informed
of the presence of the calling application. One possible solution to
this would be to allow users to write module initialization code in
module-info.java. Then an application coder would have a solid
reliable place to put code that registers the additional configuration
files with the low-level library. Something like:

module com.foo.app {
  requires org.threeten.extra;

  init(ModuleInitContext context) {
UtcRules.registerLeapSecondFile("/com/foo/app/LeapSeconds.txt");
  }
}

PS. ServiceLoader is a pain to use in Java 9 too. As a library doesn't
know whether it will run as a named module or on the classpath, I have
to duplicate the service loader configuration - once in
META-INF/services and once in module-info.java, which is horrible. It
also means that the provide() static method is a feature that cannot
be used by libraries.

Stephen

Re: ClassLoader.getResources(String)

[Alan Bateman]

On 07/03/2018 12:11, Stephen Colebourne wrote:

Following up on this, it does feel like the use case is now simply not
possible. I have a similar problem with
ClassLoader.getResources(String) in threeten-extra.

https://github.com/ThreeTen/threetenbp-extra/blob/master/src/main/java/org/threeten/extra/scale/SystemUTCRules.java#L202

The ThreeTen-Extra project defines a config file
org/threeten/extra/scale/LeapSecond.txt. The code uses
ClassLoader.getResources(String) to find the latest version of the
file, which may be in the threeten-extra jar file, or in any jar file
that uses threeten-extra.jar. ie. to replace the version from
threeten-extra.jar, a user simply has to add a file with the same
name/package to their jar file.

threeten-extra.jar contains org/threeten/extra/scale/LeapSecond.txt
application.jar also contains org/threeten/extra/scale/LeapSecond.txt

Under JPMS this fails, as the resource cannot be located in
org/threeten/extra/scale in a different jar file. But this appears
makes the whole design impossible to make work with JPMS.

The code in threeten-extra.jar cannot possibly know about the package
names of the application.jar, so there is no way for it to find the
config file.

There seem to be only two solutions to this
- ServiceLoader, but that is for code, not config files
- forcing the application to manually register their config file

Both of these provide a markedly worse outcome.

Am I missing something?

Resources can't be both encapsulated and not encapsulated at the same 
time. If a module has a resource in a package that is intended to be 
located by code in other modules using ClassLoader getResourceXXX then 
it has to open the package.


In the above, then I assume the main issue isn't resource encapsulation, 
it's that you've got two modules on the application module class 
containing the same package so they can't both be mapped to the 
application class loader.


You've dismissed services but I would expect it to provide a nice 
solution. The service interface might be very simple, something like:


public interface LeapSecondDataProvider {
    LeapSecondData data();
}

or better still, define methods that allow SystemUTCRules select the 
right version of the leap second data. Applications that ship their own 
leap data second would ship an implementation of this class. Yes, it's 
different to searching the class file for configuration files but a lot 
more reliable.


-Alan

Re: java.beans package in java.desktop module

[Remi Forax]

- Mail original -
> De: "Stephen Colebourne" 
> À: "jigsaw-dev" 
> Envoyé: Mercredi 7 Mars 2018 13:23:10
> Objet: Re: java.beans package in java.desktop module

> What is needed is an abstraction that can work for all sorts of
> data-like classes
> 
> - classic JavaBeans
> - records
> - value types
> - HashMaps
> - JSON objects
> - etc
> 
> Its not rocket science as an API, but has been needed for many years
> as so many projects have this code duplicated (leading to lots of
> subtle differences).
> 
> The API cannot be based on method handles, as in the HashMap case
> there is no property-specific method to call.

you can bind [1] (do partial evaluation of) Map::get with the property name,

>  But there is no reason why the implementation of the interface for records 
> could not use
> method handles internally.

apart if valhalla generics are around, such interface will require to box and 
unbox values.

> 
> Stephen

Rémi

> 
> 
> On 7 March 2018 at 11:21, Remi Forax  wrote:
>> As Stephen said, with the introduction of the Pattern Matching in the near
>> future, an API to extract the values from an object (the de-constructor API) 
>> or
>> at least from a record object will have to be created, but it may be based on
>> method handles, so perhaps not using a direct interface.
>>
>> Rémi
>>
>> - Mail original -
>>> De: "Guillaume Smet" 
>>> À: "Stephen Colebourne" 
>>> Cc: "jigsaw-dev" 
>>> Envoyé: Mardi 6 Mars 2018 18:45:21
>>> Objet: Re: java.beans package in java.desktop module
>>
>>> Hi Stephen,
>>>
>>> On Tue, Mar 6, 2018 at 3:29 PM, Stephen Colebourne 
>>> wrote:
>>>
 It had been my hope that we might see a replacement for the java.beans
 package. I drew up a rough prototype here:
  https://github.com/jodastephen/property-alliance
 based on previous work in Joda-Beans. More info here:
  http://jodastephen.github.io/property-alliance/

 If an interface-based design were adopted, it could be implemented by
 the existing JavaBeans code and also by future language changes such
 as records (data classes), while the API could be used far more
 broadly, such as in Hibernate or EL.

 Sadly, I haven't had the time or energy to progress this, but the need is
 there.
>>>
>>>
>>> Thanks for sharing. It indeed sound like something worth pursuing.
>>>
>>> Having a new API in the JDK indeed seems the only way out for the
>>> java.beans issue.
>>>
>>> --
> >> Guillaume

Re: java.beans package in java.desktop module

[Stephen Colebourne]

What is needed is an abstraction that can work for all sorts of
data-like classes

- classic JavaBeans
- records
- value types
- HashMaps
- JSON objects
- etc

Its not rocket science as an API, but has been needed for many years
as so many projects have this code duplicated (leading to lots of
subtle differences).

The API cannot be based on method handles, as in the HashMap case
there is no property-specific method to call. But there is no reason
why the implementation of the interface for records could not use
method handles internally.

Stephen


On 7 March 2018 at 11:21, Remi Forax  wrote:
> As Stephen said, with the introduction of the Pattern Matching in the near 
> future, an API to extract the values from an object (the de-constructor API) 
> or at least from a record object will have to be created, but it may be based 
> on method handles, so perhaps not using a direct interface.
>
> Rémi
>
> - Mail original -
>> De: "Guillaume Smet" 
>> À: "Stephen Colebourne" 
>> Cc: "jigsaw-dev" 
>> Envoyé: Mardi 6 Mars 2018 18:45:21
>> Objet: Re: java.beans package in java.desktop module
>
>> Hi Stephen,
>>
>> On Tue, Mar 6, 2018 at 3:29 PM, Stephen Colebourne 
>> wrote:
>>
>>> It had been my hope that we might see a replacement for the java.beans
>>> package. I drew up a rough prototype here:
>>>  https://github.com/jodastephen/property-alliance
>>> based on previous work in Joda-Beans. More info here:
>>>  http://jodastephen.github.io/property-alliance/
>>>
>>> If an interface-based design were adopted, it could be implemented by
>>> the existing JavaBeans code and also by future language changes such
>>> as records (data classes), while the API could be used far more
>>> broadly, such as in Hibernate or EL.
>>>
>>> Sadly, I haven't had the time or energy to progress this, but the need is
>>> there.
>>
>>
>> Thanks for sharing. It indeed sound like something worth pursuing.
>>
>> Having a new API in the JDK indeed seems the only way out for the
>> java.beans issue.
>>
>> --
>> Guillaume

Re: ClassLoader.getResources(String)

[Stephen Colebourne]

Following up on this, it does feel like the use case is now simply not
possible. I have a similar problem with
ClassLoader.getResources(String) in threeten-extra.

https://github.com/ThreeTen/threetenbp-extra/blob/master/src/main/java/org/threeten/extra/scale/SystemUTCRules.java#L202

The ThreeTen-Extra project defines a config file
org/threeten/extra/scale/LeapSecond.txt. The code uses
ClassLoader.getResources(String) to find the latest version of the
file, which may be in the threeten-extra jar file, or in any jar file
that uses threeten-extra.jar. ie. to replace the version from
threeten-extra.jar, a user simply has to add a file with the same
name/package to their jar file.

threeten-extra.jar contains org/threeten/extra/scale/LeapSecond.txt
application.jar also contains org/threeten/extra/scale/LeapSecond.txt

Under JPMS this fails, as the resource cannot be located in
org/threeten/extra/scale in a different jar file. But this appears
makes the whole design impossible to make work with JPMS.

The code in threeten-extra.jar cannot possibly know about the package
names of the application.jar, so there is no way for it to find the
config file.

There seem to be only two solutions to this
- ServiceLoader, but that is for code, not config files
- forcing the application to manually register their config file

Both of these provide a markedly worse outcome.

Am I missing something?

Stephen


On 7 February 2018 at 20:11, Alan Bateman  wrote:
> On 07/02/2018 16:56, Stephen Colebourne wrote:
>>
>> :
>> I was using maven to create a jar-with-dependencies file, so I could
>> use jlink. With all the code in one jar file, there shouldn't be any
>> access barriers to worry about.
>>
>> ClassLoader.getResources(String) worked just fine until Java 9. The
>> two APIs are not comparable - the ClassLoader one returns all URLs
>> found, whereas the Class one returns just one URL. Switching API would
>> change behaviour.
>
> ClassLoader.getResources searches the class path as it did in JDK 9 and
> older, it it just can't locate non-".class" resources in modules when they
> are encapsulated. Class loaders are oblivious as to who is ultimately
> attempting to load a class or locate a resource (the initiating and defining
> loader can be different, they can many class loaders in the delegation
> chain).
>
> With the uber modular JAR scenario then all classes for several libraries
> are in the same module. This means that the names of resources in that
> module are unique. If several libraries have the same resource then I assume
> you drop all but one when you create this uber JAR (or maybe you are merging
> some of the configuration files, I can't tell). So I assume you could change
> this code to use Class.getResource and it will locate at-most-one resource
> with a specific name.
>
> To do a proper migration means re-examining ResourceConfig of course. Using
> services is likely to be a lot cleaner and more robust than scanning for
> configuration files.
>
> -Alan

Re: java.beans package in java.desktop module

[Remi Forax]

As Stephen said, with the introduction of the Pattern Matching in the near 
future, an API to extract the values from an object (the de-constructor API) or 
at least from a record object will have to be created, but it may be based on 
method handles, so perhaps not using a direct interface.

Rémi

- Mail original -
> De: "Guillaume Smet" 
> À: "Stephen Colebourne" 
> Cc: "jigsaw-dev" 
> Envoyé: Mardi 6 Mars 2018 18:45:21
> Objet: Re: java.beans package in java.desktop module

> Hi Stephen,
> 
> On Tue, Mar 6, 2018 at 3:29 PM, Stephen Colebourne 
> wrote:
> 
>> It had been my hope that we might see a replacement for the java.beans
>> package. I drew up a rough prototype here:
>>  https://github.com/jodastephen/property-alliance
>> based on previous work in Joda-Beans. More info here:
>>  http://jodastephen.github.io/property-alliance/
>>
>> If an interface-based design were adopted, it could be implemented by
>> the existing JavaBeans code and also by future language changes such
>> as records (data classes), while the API could be used far more
>> broadly, such as in Hibernate or EL.
>>
>> Sadly, I haven't had the time or energy to progress this, but the need is
>> there.
> 
> 
> Thanks for sharing. It indeed sound like something worth pursuing.
> 
> Having a new API in the JDK indeed seems the only way out for the
> java.beans issue.
> 
> --
> Guillaume