Re: new branch for JPMS?

[Christopher Johnson]

Hi Andy and list,

Thank you for the feedback.  I have been working on the Jena JPMS
experiment and a rough concept be reviewed here:
https://github.com/christopher-johnson/jena-redux/tree/jpms

These selected modules can be built now and here is a dependency graph of
jena.arq:
https://raw.githubusercontent.com/christopher-johnson/jena-redux/jpms/docs/jena-arq-graph.png

and here is jena.dboe
https://raw.githubusercontent.com/christopher-johnson/jena-redux/jpms/docs/jena-dboe-graph.png

The refactoring strategy follows the "groupid.rootpackage" convention that
is expected for module resolution.  Since "org.apache.jena" is the group
id, ambiguity would remain for resolution within the module without a
specific root identifier.  So the above mentioned "uber jar" is probably
not possible.  Within the packages themselves, no code was actually
touched.  Only the imports were changed.  A simple migration script could
be written to help existing users change their imports.

The testing apparatus is where the majority of work remains.   Jigsaw/JPMS
does not allow test visibility between modules, so either the abstraction
has to occur either in main, or the inherited test classes need to
duplicated for each module.  In order to get the tests to compile, the
quickest option was just to duplicate the abstracted and base test
classes.  This is certainly not optimal.  The tests do run, but certainly
not without exceptions.  The EarlTest classes (like "QueryTest") also
needed to be disabled because the parameterized TestCase constructor
classes will not compile.  To discern the testing logic requires more
research and time for me.

As far as the benefits, the fact is that the entire JDK is built on modules
now.  It is the paradigm that Oracle has decided to implement and it is
here to stay (whether we like it or not).  There are a lot of good reasons
to develop towards modules and very few negatives.  Resolving a code base's
problematic dependencies can be a puzzle, but for me personally, solving
puzzles is interesting.  Remi Forax points out in this interview
 that selling Jigsaw/JPMS is difficult
because it restricts (and breaks) what can be done now, but the benefits
can be reaped in the added management and design functionality that is
problematic to achieve while trapped in "classpath hell".

Here are a few references for further information:
https://mreinhold.org/blog/jigsaw-complete
http://openjdk.java.net/projects/jigsaw/spec/sotms/

Christopher Johnson
Scientific Associate
Universitätsbibliothek Leipzig

On 27 April 2018 at 18:04, Andy Seaborne  wrote:

> Such a repacking is a "Jena 4" and if we're doing Jena4 the community will
> which so consider other changes as well.
>
> In order to experiment with JPMS, it does not need a branch in the
> codebase - you can clone and develop in a separate github repository now.
> There is no barrier.
>
> There are many users of jena and there is a significant adoption tail. A
> major version bump with incompatibilities will length the tail. The project
> has been there at Jena1->Jena2. The experience there informed how to go
> Jena2->Jena3 - there was an RDF 1.1 change so Jena3 happened as Jena went
> RDF 1.0 to RDF 1.1.
>
> And not all users are deep technical experts in Java.
>
> We have yet to see just how much JPMS is adopted and at what rate. There
> has not been time yet to see what common, good practice arises.
>
> Maybe there are other ways to get here that do not require a lot of
> change. For example - make one-uber jar of the main modules, name it
> "org.apache.jena".  That is evolutionary for users.
>
> We have automatic-module-name to help the path to JPMS. If, generally, AMN
> turns out to be impractical (and it was a point of contention in the Java
> community vote - a part of the resolution is "we will learn
> by releasing" - JPMS is not unchanging yet). or slow in adoption,
> especially in open source, I wouldn't want to waste people's time.
>
> An experiment can lead the change - a release, with all the questions on
> users@ etc, and downstream impacts, needs t be carefully timed and I for
> one would like to see the experiment first.
>
> On 27/04/18 16:03, Christopher Johnson wrote:
>
>> Hi,
>> JPMS is certainly a major change.  I believe that it offers enormous value
>> to a codebase with strong encapsulation and as such it is similar to OSGI.
>> The benefits of OSGI are well known, though the implementations are
>> fraught
>> with complexity.  Comparatively, JPMS is relatively simple.
>>
>
> I see a major issue of timing. What value does it bring to the current
> user/application base and when?
>
> Andy
>
> Retrofitting a robust platform like Jena is without question a big
>> challenge.  JPMS exposes legacy APIs, so aging dependencies (like log4j
>> 1.x
>>
>
> 
>
> Only for some logging control and that can be removed.
>
> Personal opinion: drop log4j1, use JUL for logging in tests, in commands
> and 

Re: new branch for JPMS?

[Andy Seaborne]

Such a repacking is a "Jena 4" and if we're doing Jena4 the community 
will which so consider other changes as well.


In order to experiment with JPMS, it does not need a branch in the 
codebase - you can clone and develop in a separate github repository 
now.  There is no barrier.


There are many users of jena and there is a significant adoption tail. A 
major version bump with incompatibilities will length the tail. The 
project has been there at Jena1->Jena2. The experience there informed 
how to go Jena2->Jena3 - there was an RDF 1.1 change so Jena3 happened 
as Jena went RDF 1.0 to RDF 1.1.


And not all users are deep technical experts in Java.

We have yet to see just how much JPMS is adopted and at what rate. 
There has not been time yet to see what common, good practice arises.


Maybe there are other ways to get here that do not require a lot of 
change. For example - make one-uber jar of the main modules, name it 
"org.apache.jena".  That is evolutionary for users.


We have automatic-module-name to help the path to JPMS. If, generally, 
AMN turns out to be impractical (and it was a point of contention in the 
Java community vote - a part of the resolution is "we will learn
by releasing" - JPMS is not unchanging yet). or slow in adoption, 
especially in open source, I wouldn't want to waste people's time.


An experiment can lead the change - a release, with all the questions on 
users@ etc, and downstream impacts, needs t be carefully timed and I for 
one would like to see the experiment first.


On 27/04/18 16:03, Christopher Johnson wrote:

Hi,
JPMS is certainly a major change.  I believe that it offers enormous value
to a codebase with strong encapsulation and as such it is similar to OSGI.
The benefits of OSGI are well known, though the implementations are fraught
with complexity.  Comparatively, JPMS is relatively simple.


I see a major issue of timing. What value does it bring to the current 
user/application base and when?


Andy


Retrofitting a robust platform like Jena is without question a big
challenge.  JPMS exposes legacy APIs, so aging dependencies (like log4j 1.x




Only for some logging control and that can be removed.

Personal opinion: drop log4j1, use JUL for logging in tests, in commands 
and builtin for Fuseki.   All quite simple uses. Anything complicates, 
let the user plug their choice as we already do.


Else-project I use Jena with JUL. Works nicely.
There is JUL support code to do much the same as the output for log4j1 
as currently used.


JUL = java.util.logging.




and junit4) will also need to be updated.  Maintaining separate branches
for the long term is indeed a headache.  The branch approach provides a
short term path for learning how to make the break in a major version.

Ultimately, it is a question of compliance and standardization.  Does Jena
want to "buy into" a new global standard or risk being marginalized in the
future?

>

Regarding the composition of the branch, it could start "small".   Perhaps
just with arq, base, and core.  These would be the root packages for those
modules and the ambiguous packages could be moved under them naturally.
The testing infrastructure may need a different approach (i.e. using
JUnit5).

>

Christopher

On Fri, 27 Apr 2018, 15:38 Rob Vesse,  wrote:


I am in the same boat, this sounds like it would quickly become a
maintenance headache.  We don't have the time or people to maintain more
than one branch, particularly if that branch would be a significant
refactor of the code

I am afraid that JPMS is going to kill Java open source efforts in the
same way that .Net profiles killed .Net open source.  It seems to add a lot
of complexity for developers, requires the entire ecosystem to buy in
before it is truly usable i.e. even if we refactor our modules are all our
many dependencies compliant, and doesn't seem like it adds much value to
end users at least from my perspective.

Rob

On 27/04/2018, 14:10, "ajs6f"  wrote:

 I'm not sure how this new branch is supposed to work. If it "just
contains modules compliant with the JPMS and that is tested for JDK9+" but
the current (Maven) modules don't have a one-to-one correspondence with
usable JPMS modules, where are the "modules" for this new branch coming
from?

 Are you suggesting decomposing the entire package structure of the
codebase and recomposing it in a branch? On what basis would packages be
grouped?

 ajs6f

 > On Apr 26, 2018, at 2:58 PM, Christopher Johnson <
chjohnso...@gmail.com> wrote:
 >
 > In reference to a recent PR (https://github.com/apache/jena/pull/401),
I am
 > seeking input on a process to move forward towards JDK9+ (and JPMS)
support
 > in Jena.  As consumers of jena apis, the applications that I am
developing
 > will require JPMS support going forward, so this is an important
issue for
 > my use case.
 >
 > What I have noticed is that there will be 

Re: new branch for JPMS?

[Christopher Johnson]

Hi,
JPMS is certainly a major change.  I believe that it offers enormous value
to a codebase with strong encapsulation and as such it is similar to OSGI.
The benefits of OSGI are well known, though the implementations are fraught
with complexity.  Comparatively, JPMS is relatively simple.

Retrofitting a robust platform like Jena is without question a big
challenge.  JPMS exposes legacy APIs, so aging dependencies (like log4j 1.x
and junit4) will also need to be updated.  Maintaining separate branches
for the long term is indeed a headache.  The branch approach provides a
short term path for learning how to make the break in a major version.

Ultimately, it is a question of compliance and standardization.  Does Jena
want to "buy into" a new global standard or risk being marginalized in the
future?

Regarding the composition of the branch, it could start "small".   Perhaps
just with arq, base, and core.  These would be the root packages for those
modules and the ambiguous packages could be moved under them naturally.
The testing infrastructure may need a different approach (i.e. using
JUnit5).

Christopher

On Fri, 27 Apr 2018, 15:38 Rob Vesse,  wrote:

> I am in the same boat, this sounds like it would quickly become a
> maintenance headache.  We don't have the time or people to maintain more
> than one branch, particularly if that branch would be a significant
> refactor of the code
>
> I am afraid that JPMS is going to kill Java open source efforts in the
> same way that .Net profiles killed .Net open source.  It seems to add a lot
> of complexity for developers, requires the entire ecosystem to buy in
> before it is truly usable i.e. even if we refactor our modules are all our
> many dependencies compliant, and doesn't seem like it adds much value to
> end users at least from my perspective.
>
> Rob
>
> On 27/04/2018, 14:10, "ajs6f"  wrote:
>
> I'm not sure how this new branch is supposed to work. If it "just
> contains modules compliant with the JPMS and that is tested for JDK9+" but
> the current (Maven) modules don't have a one-to-one correspondence with
> usable JPMS modules, where are the "modules" for this new branch coming
> from?
>
> Are you suggesting decomposing the entire package structure of the
> codebase and recomposing it in a branch? On what basis would packages be
> grouped?
>
> ajs6f
>
> > On Apr 26, 2018, at 2:58 PM, Christopher Johnson <
> chjohnso...@gmail.com> wrote:
> >
> > In reference to a recent PR (https://github.com/apache/jena/pull/401),
> I am
> > seeking input on a process to move forward towards JDK9+ (and JPMS)
> support
> > in Jena.  As consumers of jena apis, the applications that I am
> developing
> > will require JPMS support going forward, so this is an important
> issue for
> > my use case.
> >
> > What I have noticed is that there will be breaking changes required
> to
> > structure the code in a way that meets the JPMS "non-interference"
> > specification.[1]  A specific problem occurs when a module does not
> contain
> > a single root package (which is a typical condition in several jena
> > modules).  This results in an "ambiguous module reference" error.  To
> > resolve this ambiguity may require considerable refactoring (e.g.
> moving
> > org.apache.jena.riot to org.apache.jena.arq.riot).  I surmise that
> it is
> > not possible to make these changes and maintain compatibility for
> existing
> > consumers
> >
> > What I would like to propose is a new branch that just contains
> modules
> > compliant with the JPMS and that is tested for JDK9+.  Artifacts
> produced
> > from this branch could be distinguished from master artifacts with an
> > appropriate appendix.
> >
> > Thank you for your consideration,
> > Christopher Johnson
> > Scientific Associate
> > Universitätsbibliothek Leipzig
> >
> > [1]
> http://openjdk.java.net/projects/jigsaw/spec/reqs/#non-interference
>
>
>
>
>
>
>

Re: new branch for JPMS?

[Rob Vesse]

I am in the same boat, this sounds like it would quickly become a maintenance 
headache.  We don't have the time or people to maintain more than one branch, 
particularly if that branch would be a significant refactor of the code

I am afraid that JPMS is going to kill Java open source efforts in the same way 
that .Net profiles killed .Net open source.  It seems to add a lot of 
complexity for developers, requires the entire ecosystem to buy in before it is 
truly usable i.e. even if we refactor our modules are all our many dependencies 
compliant, and doesn't seem like it adds much value to end users at least from 
my perspective.

Rob

On 27/04/2018, 14:10, "ajs6f"  wrote:

I'm not sure how this new branch is supposed to work. If it "just contains 
modules compliant with the JPMS and that is tested for JDK9+" but the current 
(Maven) modules don't have a one-to-one correspondence with usable JPMS 
modules, where are the "modules" for this new branch coming from?

Are you suggesting decomposing the entire package structure of the codebase 
and recomposing it in a branch? On what basis would packages be grouped?

ajs6f

> On Apr 26, 2018, at 2:58 PM, Christopher Johnson  
wrote:
> 
> In reference to a recent PR (https://github.com/apache/jena/pull/401), I 
am
> seeking input on a process to move forward towards JDK9+ (and JPMS) 
support
> in Jena.  As consumers of jena apis, the applications that I am developing
> will require JPMS support going forward, so this is an important issue for
> my use case.
> 
> What I have noticed is that there will be breaking changes required to
> structure the code in a way that meets the JPMS "non-interference"
> specification.[1]  A specific problem occurs when a module does not 
contain
> a single root package (which is a typical condition in several jena
> modules).  This results in an "ambiguous module reference" error.  To
> resolve this ambiguity may require considerable refactoring (e.g. moving
> org.apache.jena.riot to org.apache.jena.arq.riot).  I surmise that it is
> not possible to make these changes and maintain compatibility for existing
> consumers
> 
> What I would like to propose is a new branch that just contains modules
> compliant with the JPMS and that is tested for JDK9+.  Artifacts produced
> from this branch could be distinguished from master artifacts with an
> appropriate appendix.
> 
> Thank you for your consideration,
> Christopher Johnson
> Scientific Associate
> Universitätsbibliothek Leipzig
> 
> [1] http://openjdk.java.net/projects/jigsaw/spec/reqs/#non-interference

Re: new branch for JPMS?

[ajs6f]

I'm not sure how this new branch is supposed to work. If it "just contains 
modules compliant with the JPMS and that is tested for JDK9+" but the current 
(Maven) modules don't have a one-to-one correspondence with usable JPMS 
modules, where are the "modules" for this new branch coming from?

Are you suggesting decomposing the entire package structure of the codebase and 
recomposing it in a branch? On what basis would packages be grouped?

ajs6f

> On Apr 26, 2018, at 2:58 PM, Christopher Johnson  
> wrote:
> 
> In reference to a recent PR (https://github.com/apache/jena/pull/401), I am
> seeking input on a process to move forward towards JDK9+ (and JPMS) support
> in Jena.  As consumers of jena apis, the applications that I am developing
> will require JPMS support going forward, so this is an important issue for
> my use case.
> 
> What I have noticed is that there will be breaking changes required to
> structure the code in a way that meets the JPMS "non-interference"
> specification.[1]  A specific problem occurs when a module does not contain
> a single root package (which is a typical condition in several jena
> modules).  This results in an "ambiguous module reference" error.  To
> resolve this ambiguity may require considerable refactoring (e.g. moving
> org.apache.jena.riot to org.apache.jena.arq.riot).  I surmise that it is
> not possible to make these changes and maintain compatibility for existing
> consumers
> 
> What I would like to propose is a new branch that just contains modules
> compliant with the JPMS and that is tested for JDK9+.  Artifacts produced
> from this branch could be distinguished from master artifacts with an
> appropriate appendix.
> 
> Thank you for your consideration,
> Christopher Johnson
> Scientific Associate
> Universitätsbibliothek Leipzig
> 
> [1] http://openjdk.java.net/projects/jigsaw/spec/reqs/#non-interference

new branch for JPMS?

[Christopher Johnson]

In reference to a recent PR (https://github.com/apache/jena/pull/401), I am
seeking input on a process to move forward towards JDK9+ (and JPMS) support
in Jena.  As consumers of jena apis, the applications that I am developing
will require JPMS support going forward, so this is an important issue for
my use case.

What I have noticed is that there will be breaking changes required to
structure the code in a way that meets the JPMS "non-interference"
specification.[1]  A specific problem occurs when a module does not contain
a single root package (which is a typical condition in several jena
modules).  This results in an "ambiguous module reference" error.  To
resolve this ambiguity may require considerable refactoring (e.g. moving
org.apache.jena.riot to org.apache.jena.arq.riot).  I surmise that it is
not possible to make these changes and maintain compatibility for existing
consumers

What I would like to propose is a new branch that just contains modules
compliant with the JPMS and that is tested for JDK9+.  Artifacts produced
from this branch could be distinguished from master artifacts with an
appropriate appendix.

Thank you for your consideration,
Christopher Johnson
Scientific Associate
Universitätsbibliothek Leipzig

[1] http://openjdk.java.net/projects/jigsaw/spec/reqs/#non-interference