Re: [Log4j] Potential simplification to java9 builds?

[Matt Sicker]

Yeah, the docs are out of date, but this was an ongoing problem from
Java 6 to 7 to 8 to 9.

On Thu, 21 May 2020 at 10:31, Ralph Goers  wrote:
>
> Actually, it was addressed in Java 9 via StackWalker. So the mechanism to 
> walk the stack changed completely between Java 8 and Java 9. I believe 
> obtaining the Process id was also made easier in Java 9.
>
> Ralph
>
> > On May 21, 2020, at 7:53 AM, Matt Sicker  wrote:
> >
> > There are API changes in Java 9+ related to internal classes we needed
> > in previous versions. There's a bit of history behind that documented
> > here: 
> > https://logging.apache.org/log4j/2.x/log4j-api/apidocs/org/apache/logging/log4j/util/StackLocator.html
> >
> > On Thu, 21 May 2020 at 09:06, Christopher  wrote:
> >>
> >> I'm curious: what is the basis for the multi-release builds in the
> >> first place? Is the code actually different for the two versions or is
> >> it just to address build-related issues?
> >>
> >> On Sun, May 17, 2020 at 4:02 PM Ralph Goers  
> >> wrote:
> >>>
> >>> I kind of doubt it. Logging runs up against this primarily because Oracle 
> >>> dropped support for sun.reflect.Reflection at the same time they added 
> >>> StackWalker. Most everything else had years to switch to a replacement.
> >>>
> >>> https://www.elastic.co/blog/elasticsearch-java-9-and-beyond 
> >>>  - Elastic 
> >>> only tests against the Java version you have chosen to compile with. They 
> >>> rely on their CI tools to run multiple builds to these the combinations.
> >>>
> >>> I have seen mentions of other components that are delivered as 
> >>> multi-release jars but as far as I can tell Log4j 2 was the first 
> >>> mainstream library to do it so many of the problems with it have been 
> >>> encountered by us.
> >>>
> >>> Ralph
> >>>
> >>>
> >>>
>  On May 17, 2020, at 11:03 AM, Volkan Yazıcı  
>  wrote:
> 
>  Maybe a naive question, but... Does anybody know how other Apache
>  projects deal with this? Do they also require multiple JDKs to be
>  present at compile time? Do they also employ `java9` directory work
>  arounds as in log4j?
> 
>  On Sat, Apr 11, 2020 at 6:39 PM Matt Sicker  wrote:
> >
> > I was playing around with the pom a little bit yesterday when I came
> > across a relatively new maven-compiler-plugin configuration element
> > called  which can be used to output the compiled
> > classes relative to META-INF/versions/N rather than the root. This
> > looks like it would likely remove the need to use
> > maven-assembly-plugin as an intermediary step.
> >
> > I found an interesting approach linked in [1] as the multi-release
> > parent strategy with source code at [2]. I attempted to refactor
> > log4j-api to use this pattern, but I couldn't figure out how to make
> > the same pattern work for test classes (which made it impossible to
> > compile log4j-api/src/test/java9).
> >
> > I'm going to continue experimenting a bit with this, but has anyone
> > tried out the more recent multi-version tooling support? We were early
> > users of some things, so I'd imagine tooling has caught up by now.
> >
> > [1]: 
> > https://maven.apache.org/plugins/maven-compiler-plugin/multirelease.html
> > [2]: 
> > https://github.com/meterware/multirelease-parent/blob/master/pom.xml
> >
> > --
> > Matt Sicker 
> 
> >>>
> >
> >
> >
> > --
> > Matt Sicker 
> >
>
>


-- 
Matt Sicker 

Re: [Log4j] Potential simplification to java9 builds?

[Ralph Goers]

Actually, it was addressed in Java 9 via StackWalker. So the mechanism to walk 
the stack changed completely between Java 8 and Java 9. I believe obtaining the 
Process id was also made easier in Java 9.

Ralph

> On May 21, 2020, at 7:53 AM, Matt Sicker  wrote:
> 
> There are API changes in Java 9+ related to internal classes we needed
> in previous versions. There's a bit of history behind that documented
> here: 
> https://logging.apache.org/log4j/2.x/log4j-api/apidocs/org/apache/logging/log4j/util/StackLocator.html
> 
> On Thu, 21 May 2020 at 09:06, Christopher  wrote:
>> 
>> I'm curious: what is the basis for the multi-release builds in the
>> first place? Is the code actually different for the two versions or is
>> it just to address build-related issues?
>> 
>> On Sun, May 17, 2020 at 4:02 PM Ralph Goers  
>> wrote:
>>> 
>>> I kind of doubt it. Logging runs up against this primarily because Oracle 
>>> dropped support for sun.reflect.Reflection at the same time they added 
>>> StackWalker. Most everything else had years to switch to a replacement.
>>> 
>>> https://www.elastic.co/blog/elasticsearch-java-9-and-beyond 
>>>  - Elastic 
>>> only tests against the Java version you have chosen to compile with. They 
>>> rely on their CI tools to run multiple builds to these the combinations.
>>> 
>>> I have seen mentions of other components that are delivered as 
>>> multi-release jars but as far as I can tell Log4j 2 was the first 
>>> mainstream library to do it so many of the problems with it have been 
>>> encountered by us.
>>> 
>>> Ralph
>>> 
>>> 
>>> 
 On May 17, 2020, at 11:03 AM, Volkan Yazıcı  
 wrote:
 
 Maybe a naive question, but... Does anybody know how other Apache
 projects deal with this? Do they also require multiple JDKs to be
 present at compile time? Do they also employ `java9` directory work
 arounds as in log4j?
 
 On Sat, Apr 11, 2020 at 6:39 PM Matt Sicker  wrote:
> 
> I was playing around with the pom a little bit yesterday when I came
> across a relatively new maven-compiler-plugin configuration element
> called  which can be used to output the compiled
> classes relative to META-INF/versions/N rather than the root. This
> looks like it would likely remove the need to use
> maven-assembly-plugin as an intermediary step.
> 
> I found an interesting approach linked in [1] as the multi-release
> parent strategy with source code at [2]. I attempted to refactor
> log4j-api to use this pattern, but I couldn't figure out how to make
> the same pattern work for test classes (which made it impossible to
> compile log4j-api/src/test/java9).
> 
> I'm going to continue experimenting a bit with this, but has anyone
> tried out the more recent multi-version tooling support? We were early
> users of some things, so I'd imagine tooling has caught up by now.
> 
> [1]: 
> https://maven.apache.org/plugins/maven-compiler-plugin/multirelease.html
> [2]: https://github.com/meterware/multirelease-parent/blob/master/pom.xml
> 
> --
> Matt Sicker 
 
>>> 
> 
> 
> 
> -- 
> Matt Sicker 
> 

Re: [Log4j] Potential simplification to java9 builds?

[Matt Sicker]

There are API changes in Java 9+ related to internal classes we needed
in previous versions. There's a bit of history behind that documented
here: 
https://logging.apache.org/log4j/2.x/log4j-api/apidocs/org/apache/logging/log4j/util/StackLocator.html

On Thu, 21 May 2020 at 09:06, Christopher  wrote:
>
> I'm curious: what is the basis for the multi-release builds in the
> first place? Is the code actually different for the two versions or is
> it just to address build-related issues?
>
> On Sun, May 17, 2020 at 4:02 PM Ralph Goers  
> wrote:
> >
> > I kind of doubt it. Logging runs up against this primarily because Oracle 
> > dropped support for sun.reflect.Reflection at the same time they added 
> > StackWalker. Most everything else had years to switch to a replacement.
> >
> > https://www.elastic.co/blog/elasticsearch-java-9-and-beyond 
> >  - Elastic 
> > only tests against the Java version you have chosen to compile with. They 
> > rely on their CI tools to run multiple builds to these the combinations.
> >
> > I have seen mentions of other components that are delivered as 
> > multi-release jars but as far as I can tell Log4j 2 was the first 
> > mainstream library to do it so many of the problems with it have been 
> > encountered by us.
> >
> > Ralph
> >
> >
> >
> > > On May 17, 2020, at 11:03 AM, Volkan Yazıcı  
> > > wrote:
> > >
> > > Maybe a naive question, but... Does anybody know how other Apache
> > > projects deal with this? Do they also require multiple JDKs to be
> > > present at compile time? Do they also employ `java9` directory work
> > > arounds as in log4j?
> > >
> > > On Sat, Apr 11, 2020 at 6:39 PM Matt Sicker  wrote:
> > >>
> > >> I was playing around with the pom a little bit yesterday when I came
> > >> across a relatively new maven-compiler-plugin configuration element
> > >> called  which can be used to output the compiled
> > >> classes relative to META-INF/versions/N rather than the root. This
> > >> looks like it would likely remove the need to use
> > >> maven-assembly-plugin as an intermediary step.
> > >>
> > >> I found an interesting approach linked in [1] as the multi-release
> > >> parent strategy with source code at [2]. I attempted to refactor
> > >> log4j-api to use this pattern, but I couldn't figure out how to make
> > >> the same pattern work for test classes (which made it impossible to
> > >> compile log4j-api/src/test/java9).
> > >>
> > >> I'm going to continue experimenting a bit with this, but has anyone
> > >> tried out the more recent multi-version tooling support? We were early
> > >> users of some things, so I'd imagine tooling has caught up by now.
> > >>
> > >> [1]: 
> > >> https://maven.apache.org/plugins/maven-compiler-plugin/multirelease.html
> > >> [2]: https://github.com/meterware/multirelease-parent/blob/master/pom.xml
> > >>
> > >> --
> > >> Matt Sicker 
> > >
> >



-- 
Matt Sicker 

Re: [Log4j] Potential simplification to java9 builds?

[Christopher]

I'm curious: what is the basis for the multi-release builds in the
first place? Is the code actually different for the two versions or is
it just to address build-related issues?

On Sun, May 17, 2020 at 4:02 PM Ralph Goers  wrote:
>
> I kind of doubt it. Logging runs up against this primarily because Oracle 
> dropped support for sun.reflect.Reflection at the same time they added 
> StackWalker. Most everything else had years to switch to a replacement.
>
> https://www.elastic.co/blog/elasticsearch-java-9-and-beyond 
>  - Elastic only 
> tests against the Java version you have chosen to compile with. They rely on 
> their CI tools to run multiple builds to these the combinations.
>
> I have seen mentions of other components that are delivered as multi-release 
> jars but as far as I can tell Log4j 2 was the first mainstream library to do 
> it so many of the problems with it have been encountered by us.
>
> Ralph
>
>
>
> > On May 17, 2020, at 11:03 AM, Volkan Yazıcı  wrote:
> >
> > Maybe a naive question, but... Does anybody know how other Apache
> > projects deal with this? Do they also require multiple JDKs to be
> > present at compile time? Do they also employ `java9` directory work
> > arounds as in log4j?
> >
> > On Sat, Apr 11, 2020 at 6:39 PM Matt Sicker  wrote:
> >>
> >> I was playing around with the pom a little bit yesterday when I came
> >> across a relatively new maven-compiler-plugin configuration element
> >> called  which can be used to output the compiled
> >> classes relative to META-INF/versions/N rather than the root. This
> >> looks like it would likely remove the need to use
> >> maven-assembly-plugin as an intermediary step.
> >>
> >> I found an interesting approach linked in [1] as the multi-release
> >> parent strategy with source code at [2]. I attempted to refactor
> >> log4j-api to use this pattern, but I couldn't figure out how to make
> >> the same pattern work for test classes (which made it impossible to
> >> compile log4j-api/src/test/java9).
> >>
> >> I'm going to continue experimenting a bit with this, but has anyone
> >> tried out the more recent multi-version tooling support? We were early
> >> users of some things, so I'd imagine tooling has caught up by now.
> >>
> >> [1]: 
> >> https://maven.apache.org/plugins/maven-compiler-plugin/multirelease.html
> >> [2]: https://github.com/meterware/multirelease-parent/blob/master/pom.xml
> >>
> >> --
> >> Matt Sicker 
> >
>

Re: [Log4j] Potential simplification to java9 builds?

[Ralph Goers]

I kind of doubt it. Logging runs up against this primarily because Oracle 
dropped support for sun.reflect.Reflection at the same time they added 
StackWalker. Most everything else had years to switch to a replacement.  

https://www.elastic.co/blog/elasticsearch-java-9-and-beyond 
 - Elastic only 
tests against the Java version you have chosen to compile with. They rely on 
their CI tools to run multiple builds to these the combinations.

I have seen mentions of other components that are delivered as multi-release 
jars but as far as I can tell Log4j 2 was the first mainstream library to do it 
so many of the problems with it have been encountered by us.

Ralph



> On May 17, 2020, at 11:03 AM, Volkan Yazıcı  wrote:
> 
> Maybe a naive question, but... Does anybody know how other Apache
> projects deal with this? Do they also require multiple JDKs to be
> present at compile time? Do they also employ `java9` directory work
> arounds as in log4j?
> 
> On Sat, Apr 11, 2020 at 6:39 PM Matt Sicker  wrote:
>> 
>> I was playing around with the pom a little bit yesterday when I came
>> across a relatively new maven-compiler-plugin configuration element
>> called  which can be used to output the compiled
>> classes relative to META-INF/versions/N rather than the root. This
>> looks like it would likely remove the need to use
>> maven-assembly-plugin as an intermediary step.
>> 
>> I found an interesting approach linked in [1] as the multi-release
>> parent strategy with source code at [2]. I attempted to refactor
>> log4j-api to use this pattern, but I couldn't figure out how to make
>> the same pattern work for test classes (which made it impossible to
>> compile log4j-api/src/test/java9).
>> 
>> I'm going to continue experimenting a bit with this, but has anyone
>> tried out the more recent multi-version tooling support? We were early
>> users of some things, so I'd imagine tooling has caught up by now.
>> 
>> [1]: https://maven.apache.org/plugins/maven-compiler-plugin/multirelease.html
>> [2]: https://github.com/meterware/multirelease-parent/blob/master/pom.xml
>> 
>> --
>> Matt Sicker 
> 

Re: [Log4j] Potential simplification to java9 builds?

[Matt Sicker]

Maven and Ant are a couple places that likely needed custom solutions,
but probably not applicable here. Maybe something in Commons? I tried
looking at a few random projects, but I'm not seeing anything. Spring
Framework might have something.

On Sun, 17 May 2020 at 13:03, Volkan Yazıcı  wrote:
>
> Maybe a naive question, but... Does anybody know how other Apache
> projects deal with this? Do they also require multiple JDKs to be
> present at compile time? Do they also employ `java9` directory work
> arounds as in log4j?
>
> On Sat, Apr 11, 2020 at 6:39 PM Matt Sicker  wrote:
> >
> > I was playing around with the pom a little bit yesterday when I came
> > across a relatively new maven-compiler-plugin configuration element
> > called  which can be used to output the compiled
> > classes relative to META-INF/versions/N rather than the root. This
> > looks like it would likely remove the need to use
> > maven-assembly-plugin as an intermediary step.
> >
> > I found an interesting approach linked in [1] as the multi-release
> > parent strategy with source code at [2]. I attempted to refactor
> > log4j-api to use this pattern, but I couldn't figure out how to make
> > the same pattern work for test classes (which made it impossible to
> > compile log4j-api/src/test/java9).
> >
> > I'm going to continue experimenting a bit with this, but has anyone
> > tried out the more recent multi-version tooling support? We were early
> > users of some things, so I'd imagine tooling has caught up by now.
> >
> > [1]: 
> > https://maven.apache.org/plugins/maven-compiler-plugin/multirelease.html
> > [2]: https://github.com/meterware/multirelease-parent/blob/master/pom.xml
> >
> > --
> > Matt Sicker 



-- 
Matt Sicker 

Re: [Log4j] Potential simplification to java9 builds?

[Volkan Yazıcı]

Maybe a naive question, but... Does anybody know how other Apache
projects deal with this? Do they also require multiple JDKs to be
present at compile time? Do they also employ `java9` directory work
arounds as in log4j?

On Sat, Apr 11, 2020 at 6:39 PM Matt Sicker  wrote:
>
> I was playing around with the pom a little bit yesterday when I came
> across a relatively new maven-compiler-plugin configuration element
> called  which can be used to output the compiled
> classes relative to META-INF/versions/N rather than the root. This
> looks like it would likely remove the need to use
> maven-assembly-plugin as an intermediary step.
>
> I found an interesting approach linked in [1] as the multi-release
> parent strategy with source code at [2]. I attempted to refactor
> log4j-api to use this pattern, but I couldn't figure out how to make
> the same pattern work for test classes (which made it impossible to
> compile log4j-api/src/test/java9).
>
> I'm going to continue experimenting a bit with this, but has anyone
> tried out the more recent multi-version tooling support? We were early
> users of some things, so I'd imagine tooling has caught up by now.
>
> [1]: https://maven.apache.org/plugins/maven-compiler-plugin/multirelease.html
> [2]: https://github.com/meterware/multirelease-parent/blob/master/pom.xml
>
> --
> Matt Sicker 

Re: [Log4j] Potential simplification to java9 builds?

[Matt Sicker]

Yikes, that sounds similar to OSGi. Adding modules makes testing into more
of an integration exercise since traditional JUnit usage assumes a flat
class path.

On Sun, May 17, 2020 at 00:10 Ralph Goers 
wrote:

> I played around with this today. I don’t think it is possible to make the
> tests work.  The reason is that from what I understand the multi-release
> format only works when the classes are packaged in a jar, which hasn’t
> happened when the tests are compiled. This causes the compile to fail since
> it can’t find the Java 9+ classes.
>
> The only way around that would be to package the files in a jar before
> running the tests, which is opposite of what Maven is trying to do.
>
> Ralph
>
> > On Apr 11, 2020, at 9:39 AM, Matt Sicker  wrote:
> >
> > I was playing around with the pom a little bit yesterday when I came
> > across a relatively new maven-compiler-plugin configuration element
> > called  which can be used to output the compiled
> > classes relative to META-INF/versions/N rather than the root. This
> > looks like it would likely remove the need to use
> > maven-assembly-plugin as an intermediary step.
> >
> > I found an interesting approach linked in [1] as the multi-release
> > parent strategy with source code at [2]. I attempted to refactor
> > log4j-api to use this pattern, but I couldn't figure out how to make
> > the same pattern work for test classes (which made it impossible to
> > compile log4j-api/src/test/java9).
> >
> > I'm going to continue experimenting a bit with this, but has anyone
> > tried out the more recent multi-version tooling support? We were early
> > users of some things, so I'd imagine tooling has caught up by now.
> >
> > [1]:
> https://maven.apache.org/plugins/maven-compiler-plugin/multirelease.html
> > [2]:
> https://github.com/meterware/multirelease-parent/blob/master/pom.xml
> >
> > --
> > Matt Sicker 
> >
>
>
> --
Matt Sicker 

Re: [Log4j] Potential simplification to java9 builds?

[Ralph Goers]

I played around with this today. I don’t think it is possible to make the tests 
work.  The reason is that from what I understand the multi-release format only 
works when the classes are packaged in a jar, which hasn’t happened when the 
tests are compiled. This causes the compile to fail since it can’t find the 
Java 9+ classes.

The only way around that would be to package the files in a jar before running 
the tests, which is opposite of what Maven is trying to do.

Ralph

> On Apr 11, 2020, at 9:39 AM, Matt Sicker  wrote:
> 
> I was playing around with the pom a little bit yesterday when I came
> across a relatively new maven-compiler-plugin configuration element
> called  which can be used to output the compiled
> classes relative to META-INF/versions/N rather than the root. This
> looks like it would likely remove the need to use
> maven-assembly-plugin as an intermediary step.
> 
> I found an interesting approach linked in [1] as the multi-release
> parent strategy with source code at [2]. I attempted to refactor
> log4j-api to use this pattern, but I couldn't figure out how to make
> the same pattern work for test classes (which made it impossible to
> compile log4j-api/src/test/java9).
> 
> I'm going to continue experimenting a bit with this, but has anyone
> tried out the more recent multi-version tooling support? We were early
> users of some things, so I'd imagine tooling has caught up by now.
> 
> [1]: https://maven.apache.org/plugins/maven-compiler-plugin/multirelease.html
> [2]: https://github.com/meterware/multirelease-parent/blob/master/pom.xml
> 
> -- 
> Matt Sicker 
>