Re: JPMS, Maven and test jars
> On Jun 22, 2021, at 2:29 PM, Andres Almiray wrote: > > Ralph, > > Once you move test code to their own Maven module they become a top level > module, that is, they have a life of their own as any other top level module. > > In contrast -tests jars are not top level, but a “variant” (that’s why they > have a classifier) of their owning top level module. > > Test jars must be published to a repository in order to be consumed, just > like any other jar, variant or not. The only difference (at least that I can > see) is that test jars may be downloaded using the coordinates of the owning > module (plus test classifier) and their release cycle is tightly coupled with > their owning module. > > But if you take away the owning relationship you’re left with, well, just > another regular Maven module. > > Personally I prefer the use of explicit dependencies and modules, barely make > use of test jar support, but I do recognize others feel differently about > them and prefer the workflow enables by such jars and conventions Well, I see that Maven now recommends not generating jars with the tests classifier - https://maven.apache.org/plugins/maven-jar-plugin/examples/create-test-jar.html. I am not sure when that was changed to be the recommended approach but given that it is I can at least give it a shot. I’m still concerned that the Maven compiler plugin won’t understand what is happening but I can only determine that by testing it. Ralph - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: JPMS, Maven and test jars
Ralph, Once you move test code to their own Maven module they become a top level module, that is, they have a life of their own as any other top level module. In contrast -tests jars are not top level, but a “variant” (that’s why they have a classifier) of their owning top level module. Test jars must be published to a repository in order to be consumed, just like any other jar, variant or not. The only difference (at least that I can see) is that test jars may be downloaded using the coordinates of the owning module (plus test classifier) and their release cycle is tightly coupled with their owning module. But if you take away the owning relationship you’re left with, well, just another regular Maven module. Personally I prefer the use of explicit dependencies and modules, barely make use of test jar support, but I do recognize others feel differently about them and prefer the workflow enables by such jars and conventions Cheers Andres Sent from my primitive tricorder > On 22 Jun 2021, at 23:20, Ralph Goers wrote: > > > >> On Jun 22, 2021, at 11:16 AM, Robert Scholte wrote: >> >> If I understand correctly you want both log4j-core main classes and test >> classes be distributed as modularized jars. >> To me with JPMS you should move the modularized test-classes to a separate >> Maven module as a first citizen artifact. >> That should make it a lot easier. > > Robert, > > I have a question on the above. I am generating log4j-core-3.0.0-SNAPSHOT.jar > and log4j-core-3.0.0-SNAPSHOT-tests.jar. > If the source is left in log4j-core and I create a log4j-core-tests module > what would I specify in the pom to get it to create the > test jar as log4j-core-3.0.0-SNAPSHOT-tests.jar? I would not want it to be > log4j-core-tests-3.0.0-SNAPSHOT.jar as that > would not be understood to be a test jar. > > Ralph > > > > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: JPMS, Maven and test jars
> On Jun 22, 2021, at 11:16 AM, Robert Scholte wrote: > > If I understand correctly you want both log4j-core main classes and test > classes be distributed as modularized jars. > To me with JPMS you should move the modularized test-classes to a separate > Maven module as a first citizen artifact. > That should make it a lot easier. Robert, I have a question on the above. I am generating log4j-core-3.0.0-SNAPSHOT.jar and log4j-core-3.0.0-SNAPSHOT-tests.jar. If the source is left in log4j-core and I create a log4j-core-tests module what would I specify in the pom to get it to create the test jar as log4j-core-3.0.0-SNAPSHOT-tests.jar? I would not want it to be log4j-core-tests-3.0.0-SNAPSHOT.jar as that would not be understood to be a test jar. Ralph - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: JPMS, Maven and test jars
That's just broken, I can have any number of package private classes (or methods) that perform lower level or non-customer appropriate tasks but that are large enough or complex enough to merrit unit testing on their own. Only testing public APIs is saying that blackboxing is the only way to test and makes whitebox testing impossible. Less testing is not more brittle. Bummer. Gary On Tue, Jun 22, 2021, 16:20 Elliotte Rusty Harold wrote: > On Tue, Jun 22, 2021 at 7:32 PM Gary Gregory > wrote: > > > In addition, how is one supposed to test package private code if a test > jar > > tests a main jar where all packages are now different? > > Some projects adhere to strictly testing through the public API only. > It definitely makes tests more robust and less brittle. > > > > -- > Elliotte Rusty Harold > elh...@ibiblio.org > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > >
Re: JPMS, Maven and test jars
Ralph, Apologies. I did not link to Jipsy as a suggestion to be used as a dependency but rather as an example of how the ModiTect plugin may be applied to a project that produces and tests an APT. I believe Jipsy has a 2nd Maven module that produces a “test” jar without it being an actual test jar, that is, with the -tests classifier. This is the approach mentioned by Robert. ModiTect helps you keep a codebase compatible with Java 8 while also supporting JPMS with full module descriptors added to the final jar as an MRJAR. Hope this helps. Cheers Andres Sent from my primitive tricorder > On 22 Jun 2021, at 22:42, Ralph Goers wrote: > > Andres, > > I looked at moditect when I first started doing the work. I’m afraid I am not > understanding how > it could help solve the problem with the test jar and unit tests. Could you > please explain that? > > I am also not sure I understand how Jipsy will help. Log4j generates the file > in META-INF/services > when it generates the service class. It does not add the provides statement > to module-info.java. > Perhaps if it also did that it would bypass the compiler error saying the > service can’t be found. > > That is certainly worth looking into. > > Ralph > > > >> On Jun 22, 2021, at 12:41 AM, Andres Almiray wrote: >> >> Have you tried using the ModiTect Maven plugin? >> >> >> https://github.com/moditect/moditect >> >> There are many ways to use it: >> - with explicit module-info.java >> - with an embedded DSL in the pom.xml >> - with an embedded module descriptor in the pom.xml >> >> Ii use the last option on Jipsy [https://github.com/kordamp/jipsy/], which >> happens to define an APT as well. >> >> https://github.com/kordamp/jipsy/blob/master/jipsy-processor/pom.xml >> >> Cheers, >> Andres >> >> --- >> Java Champion; Groovy Enthusiast >> http://andresalmiray.com >> http://www.linkedin.com/in/aalmiray >> -- >> What goes up, must come down. Ask any system administrator. >> There are 10 types of people in the world: Those who understand binary, and >> those who don't. >> To understand recursion, we must first understand recursion. >> >> >>> On Tue, Jun 22, 2021 at 7:02 AM Ralph Goers >>> wrote: >>> >>> Sorry for posting again. I really need to proof-read better. Please ignore >>> the prior email. >>> >>> I have recently had quite an adventure modifying several of Log4j’s Maven >>> modules to >>> implement JPMS on our master branch. It was an adventure due to a few >>> issues: >>> >>> 1. https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826 < >>> https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826>. >>> This bug has been fixed in Java 17 but Log4j uses Java 11 to build. >>> 2. Log4j-plugins builds an annotation processor, packages it with the >>> annotations >>> and classes necessary to build and run plugins, creates and test jar >>> and runs unit tests. >>> 3. It is not possible to compile an annotation processor with a >>> module-info.java present. >>> The compile will fail because it can’t find the annotation processor >>> “service” when >>> compiling module-info.java. >>> 4. It is very difficult to compile an annotation processor and then use it >>> in the same Maven >>> module. JPMS expects the annotation processor to either be on the >>> classpath or specified >>> with the processorpath option. When a module-info.java is present, >>> Maven automatically >>> moves everything to the module path. Maven only supports using >>> coordinates to specify the >>> processor path, which don’t exist when the processor is in the same >>> module. The only way >>> to solve this is to compile all the classes with proc=only without a >>> module-info.java present. >>> 5. Although number 4 might seem bad, it really doesn’t matter because >>> javac will fail if a >>> module-info.java is present because module-info.java will have a >>> reference to the service >>> class being generated by the annotation processor and for some reason >>> module-info.java >>> is resolved before the annotation processor runs. >>> 6. If the main set of classes are compiled with a module-info.java every >>> other compile >>> in the Maven module must also be modularized. Likewise, if the main >>> module does >>> not contain a module-info.java no other compile can either. >>> 7. JPMS requires that every module have its own package space with no >>> overlap >>> with any other JPMS module. >>> >>> So while generating the log4j-plugins module is quite painful, generating >>> log4j-core isn’t >>> much better. That is actually the primary focus for this list. >>> >>> Log4j-core consists of the main classes packaged in the jar, a test jar >>> that is used by >>> downstream Maven modules, and the unit tests. Prior to JPMS one would just >>> create >>> the main jar and then package all the test classes and unit tests in a >>> test jar. This can >>> no longer be done with JPMS. >>>
Re: JPMS, Maven and test jars
Andres, I looked at moditect when I first started doing the work. I’m afraid I am not understanding how it could help solve the problem with the test jar and unit tests. Could you please explain that? I am also not sure I understand how Jipsy will help. Log4j generates the file in META-INF/services when it generates the service class. It does not add the provides statement to module-info.java. Perhaps if it also did that it would bypass the compiler error saying the service can’t be found. That is certainly worth looking into. Ralph > On Jun 22, 2021, at 12:41 AM, Andres Almiray wrote: > > Have you tried using the ModiTect Maven plugin? > > > https://github.com/moditect/moditect > > There are many ways to use it: > - with explicit module-info.java > - with an embedded DSL in the pom.xml > - with an embedded module descriptor in the pom.xml > > Ii use the last option on Jipsy [https://github.com/kordamp/jipsy/], which > happens to define an APT as well. > > https://github.com/kordamp/jipsy/blob/master/jipsy-processor/pom.xml > > Cheers, > Andres > > --- > Java Champion; Groovy Enthusiast > http://andresalmiray.com > http://www.linkedin.com/in/aalmiray > -- > What goes up, must come down. Ask any system administrator. > There are 10 types of people in the world: Those who understand binary, and > those who don't. > To understand recursion, we must first understand recursion. > > > On Tue, Jun 22, 2021 at 7:02 AM Ralph Goers > wrote: > >> Sorry for posting again. I really need to proof-read better. Please ignore >> the prior email. >> >> I have recently had quite an adventure modifying several of Log4j’s Maven >> modules to >> implement JPMS on our master branch. It was an adventure due to a few >> issues: >> >> 1. https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826 < >> https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826>. >> This bug has been fixed in Java 17 but Log4j uses Java 11 to build. >> 2. Log4j-plugins builds an annotation processor, packages it with the >> annotations >> and classes necessary to build and run plugins, creates and test jar >> and runs unit tests. >> 3. It is not possible to compile an annotation processor with a >> module-info.java present. >> The compile will fail because it can’t find the annotation processor >> “service” when >> compiling module-info.java. >> 4. It is very difficult to compile an annotation processor and then use it >> in the same Maven >> module. JPMS expects the annotation processor to either be on the >> classpath or specified >> with the processorpath option. When a module-info.java is present, >> Maven automatically >> moves everything to the module path. Maven only supports using >> coordinates to specify the >> processor path, which don’t exist when the processor is in the same >> module. The only way >> to solve this is to compile all the classes with proc=only without a >> module-info.java present. >> 5. Although number 4 might seem bad, it really doesn’t matter because >> javac will fail if a >> module-info.java is present because module-info.java will have a >> reference to the service >> class being generated by the annotation processor and for some reason >> module-info.java >> is resolved before the annotation processor runs. >> 6. If the main set of classes are compiled with a module-info.java every >> other compile >> in the Maven module must also be modularized. Likewise, if the main >> module does >> not contain a module-info.java no other compile can either. >> 7. JPMS requires that every module have its own package space with no >> overlap >>with any other JPMS module. >> >> So while generating the log4j-plugins module is quite painful, generating >> log4j-core isn’t >> much better. That is actually the primary focus for this list. >> >> Log4j-core consists of the main classes packaged in the jar, a test jar >> that is used by >> downstream Maven modules, and the unit tests. Prior to JPMS one would just >> create >> the main jar and then package all the test classes and unit tests in a >> test jar. This can >> no longer be done with JPMS. >> >> When a project publishes a test jar along with the main jar, just like any >> other JPMS >> module. the test jar cannot use the package space of the main classes. >> Log4j core >> uses org.apache.logging.log4j.core so I placed all the test utility >> classes under >> org.apache.logging.log4j.core.test. However, the unit tests all need to be >> packaged >> in the main package space since its module-info.java “extends” the main >> module and >> several unit tests are required to be in the same main package so that >> they can access >> package private methods specifically provided for testing. >> >> In order to get this to work I had to perform the following steps: >> >>• Compile all the main classes except module-info.java with the >> Plugin annotation processor.
Re: JPMS, Maven and test jars
Gary, the test jar doesn’t perform tests. It provides test utility classes like LoggerContextRule. > On Jun 22, 2021, at 12:31 PM, Gary Gregory wrote: > > It feels to me like JPMS just plainly breaks the informal industry standard > of the Maven project layout we have all been using for a million years. > > In addition, how is one supposed to test package private code if a test jar > tests a main jar where all packages are now different? This seems like a > problem whether the test code is in the same Maven module or not. Or am > I missing something? Am I the only one feeling like JPMS what developed in > a vaccum? > > Gary > Gary, testing package private stuff is fine and easy if you don’t generate a test jar. In that case unit tests behave as they always have except that you have to create a module-info.java that gives you access to the internals of the module. The problem occurs simply because the test jar being created cannot contain classes in the same package space as the main module, so you can no longer just publish your test utility classes combined with the unit tests as the test jar. Ralph - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: JPMS, Maven and test jars
> On Jun 22, 2021, at 1:19 PM, Elliotte Rusty Harold wrote: > > On Tue, Jun 22, 2021 at 7:32 PM Gary Gregory wrote: > >> In addition, how is one supposed to test package private code if a test jar >> tests a main jar where all packages are now different? > > Some projects adhere to strictly testing through the public API only. > It definitely makes tests more robust and less brittle. I completely disagree with this. There are tons of things I would never be able to test in Log4j without being able to access the internals. You simply can’t get high enough code coverage if you are limited to the public API. Ralph - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: JPMS, Maven and test jars
> On Jun 22, 2021, at 12:01 PM, Benjamin Marwell wrote: > >> So now you would have a maven module with only the main source and no tests, >> and a second module >> where the “main” source is actually the test classes >> and the unit tests test the code in a different Maven >> module with a module info that opens the module-info of the main source jar. >> Seems downright weird to me. > > Well, it is not. First of all, you can test your main code nicely > using standard non-modularized tests. > No problem here. Yes, there is a problem here. First, I am pretty sure that if I didn’t have a module-info.java file in my unit tests the build failed. Even if I could do that I wouldn’t be verifying that the module was constructed correctly. > > Then you can use your module containing the abstract/common test > classes with the scope test. > This is actually what junit etc. is – you are using their main classes > in YOUR test scope. > So, nothing weird here, too. > > The third project pulls them in altogether and is modularized. > Call it "IT" or "integration test", and everything feels normal again. So you would propose that any time someone wants to publish a test jar that they must have 3 Maven modules. Obviously I wouldn’t call it IT since it doesn’t contain integration tests, it contains unit tests. If I also had integration tests then I would need a fourth Maven module. That feels very “un-normal” to me. I don’t recall ever seeing a project that has the unit tests in a module separate from the main source. It would make more sense to me to follow Robert’s suggestion of having the test jar and unit tests in a second module, but it feels strange that the unit tests in that module test the source from a different module. I haven’t tried, but I am also not sure if Maven will behave the same way when operating on a packaged jar as it does when the module-info.java for the unit tests extends the module-info for the main classes. I’d have to test that. I have a concern that it will think it is extending the test classes JPMS module. I’ve also not tested a Maven module that has only unit tests. I suspect in that case it will not be possible to test the main source as a JPMS module since Maven seems to require a module-info.java in the main source before it will allow the unit tests to include one. > > This will also help your IDE recognize the code. > And as this is a common pattern, it will also help other coders to > more easily recognize the structure of your project. Yes, this will obviously make the IDE happier. I’m not sure about the developers though. Ralph > > Am Di., 22. Juni 2021 um 20:46 Uhr schrieb Ralph Goers > : >> >> Yes, you understand correctly. And yes, the test classes could be moved to a >> separate maven module, but >> then the unit tests would also have to move as well as they are dependent on >> those test classes. Of course, >> the test classes are dependent on the main source. As an example, log4j-core >> has a LoggerContext class, >> the test jar has a LoggerContextRule class that uses the LoggerContext and >> the LoggerContextRule is used >> by many unit tests. >> >> So now you would have a maven module with only the main source and no tests, >> and a second module >> where the “main” source is actually the test classes and the unit tests test >> the code in a different Maven >> module with a module info that opens the module-info of the main source jar. >> >> Seems downright weird to me. >> >> Yes, I realize this doesn’t fit in the normal Maven lifecycle. I brought it >> up because if and/or when more >> stuff starts becoming JPMS modularized this issue is going to become more >> and more common. >> >> To be honest, I’d be OK living with the build the way it is except people >> are complaining that their IDE >> has no idea what to do with the project. >> >> Ralph >> >>> On Jun 22, 2021, at 11:16 AM, Robert Scholte wrote: >>> >>> If I understand correctly you want both log4j-core main classes and test >>> classes be distributed as modularized jars. >>> To me with JPMS you should move the modularized test-classes to a separate >>> Maven module as a first citizen artifact. >>> That should make it a lot easier. >>> It should also make the separation more clear: classes under >>> target/test-classes are there to unittest its main classes. >>> Whereas this new module contains reusable (main) code for testing (which >>> should be tested too ;) ) >>> I understand that this will change the GA(-classifier) for the testing >>> module, but to me for a good reason. >>> >>> Even with your very small reduction of steps to simplify the process, it >>> doesn't fit in the default lifecycle unless you write you introduce a new >>> lifecycle-binding or write a new extension. >>> >>> Robert >>> >>> On 22-6-2021 07:02:59, Ralph Goers wrote: >>> Sorry for posting again. I really need to proof-read better. Please ignore >>> the prior email.
Re: JPMS, Maven and test jars
On Tue, Jun 22, 2021 at 7:32 PM Gary Gregory wrote: > In addition, how is one supposed to test package private code if a test jar > tests a main jar where all packages are now different? Some projects adhere to strictly testing through the public API only. It definitely makes tests more robust and less brittle. -- Elliotte Rusty Harold elh...@ibiblio.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: JPMS, Maven and test jars
It feels to me like JPMS just plainly breaks the informal industry standard of the Maven project layout we have all been using for a million years. In addition, how is one supposed to test package private code if a test jar tests a main jar where all packages are now different? This seems like a problem whether the test code is in the same Maven module or not. Or am I missing something? Am I the only one feeling like JPMS what developed in a vaccum? Gary On Tue, Jun 22, 2021, 14:16 Robert Scholte wrote: > If I understand correctly you want both log4j-core main classes and test > classes be distributed as modularized jars. > To me with JPMS you should move the modularized test-classes to a separate > Maven module as a first citizen artifact. > That should make it a lot easier. > It should also make the separation more clear: classes under > target/test-classes are there to unittest its main classes. > Whereas this new module contains reusable (main) code for testing (which > should be tested too ;) ) > I understand that this will change the GA(-classifier) for the testing > module, but to me for a good reason. > > Even with your very small reduction of steps to simplify the process, it > doesn't fit in the default lifecycle unless you write you introduce a new > lifecycle-binding or write a new extension. > > Robert > > On 22-6-2021 07:02:59, Ralph Goers wrote: > Sorry for posting again. I really need to proof-read better. Please ignore > the prior email. > > I have recently had quite an adventure modifying several of Log4j’s Maven > modules to > implement JPMS on our master branch. It was an adventure due to a few > issues: > > 1. https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826 . > This bug has been fixed in Java 17 but Log4j uses Java 11 to build. > 2. Log4j-plugins builds an annotation processor, packages it with the > annotations > and classes necessary to build and run plugins, creates and test jar and > runs unit tests. > 3. It is not possible to compile an annotation processor with a > module-info.java present. > The compile will fail because it can’t find the annotation processor > “service” when > compiling module-info.java. > 4. It is very difficult to compile an annotation processor and then use it > in the same Maven > module. JPMS expects the annotation processor to either be on the > classpath or specified > with the processorpath option. When a module-info.java is present, Maven > automatically > moves everything to the module path. Maven only supports using coordinates > to specify the > processor path, which don’t exist when the processor is in the same > module. The only way > to solve this is to compile all the classes with proc=only without a > module-info.java present. > 5. Although number 4 might seem bad, it really doesn’t matter because > javac will fail if a > module-info.java is present because module-info.java will have a reference > to the service > class being generated by the annotation processor and for some reason > module-info.java > is resolved before the annotation processor runs. > 6. If the main set of classes are compiled with a module-info.java every > other compile > in the Maven module must also be modularized. Likewise, if the main module > does > not contain a module-info.java no other compile can either. > 7. JPMS requires that every module have its own package space with no > overlap > with any other JPMS module. > > So while generating the log4j-plugins module is quite painful, generating > log4j-core isn’t > much better. That is actually the primary focus for this list. > > Log4j-core consists of the main classes packaged in the jar, a test jar > that is used by > downstream Maven modules, and the unit tests. Prior to JPMS one would just > create > the main jar and then package all the test classes and unit tests in a > test jar. This can > no longer be done with JPMS. > > When a project publishes a test jar along with the main jar, just like any > other JPMS > module. the test jar cannot use the package space of the main classes. > Log4j core > uses org.apache.logging.log4j.core so I placed all the test utility > classes under > org.apache.logging.log4j.core.test. However, the unit tests all need to be > packaged > in the main package space since its module-info.java “extends” the main > module and > several unit tests are required to be in the same main package so that > they can access > package private methods specifically provided for testing. > > In order to get this to work I had to perform the following steps: > > • Compile all the main classes except module-info.java with the Plugin > annotation processor. > • Compile the main module-info.java. > • Compile the test classes used by other modules with module-info.java and > using the plugin preprocessor. > • Package these test classes in a test jar. > • Delete the module-info and generated source for the test classes. > • Move the main module-info to a temp location. > • Compile
Re: JPMS, Maven and test jars
> So now you would have a maven module with only the main source and no tests, > and a second module > where the “main” source is actually the test classes > and the unit tests test the code in a different Maven > module with a module info that opens the module-info of the main source jar. > Seems downright weird to me. Well, it is not. First of all, you can test your main code nicely using standard non-modularized tests. No problem here. Then you can use your module containing the abstract/common test classes with the scope test. This is actually what junit etc. is – you are using their main classes in YOUR test scope. So, nothing weird here, too. The third project pulls them in altogether and is modularized. Call it "IT" or "integration test", and everything feels normal again. This will also help your IDE recognize the code. And as this is a common pattern, it will also help other coders to more easily recognize the structure of your project. Am Di., 22. Juni 2021 um 20:46 Uhr schrieb Ralph Goers : > > Yes, you understand correctly. And yes, the test classes could be moved to a > separate maven module, but > then the unit tests would also have to move as well as they are dependent on > those test classes. Of course, > the test classes are dependent on the main source. As an example, log4j-core > has a LoggerContext class, > the test jar has a LoggerContextRule class that uses the LoggerContext and > the LoggerContextRule is used > by many unit tests. > > So now you would have a maven module with only the main source and no tests, > and a second module > where the “main” source is actually the test classes and the unit tests test > the code in a different Maven > module with a module info that opens the module-info of the main source jar. > > Seems downright weird to me. > > Yes, I realize this doesn’t fit in the normal Maven lifecycle. I brought it > up because if and/or when more > stuff starts becoming JPMS modularized this issue is going to become more and > more common. > > To be honest, I’d be OK living with the build the way it is except people are > complaining that their IDE > has no idea what to do with the project. > > Ralph > > > On Jun 22, 2021, at 11:16 AM, Robert Scholte wrote: > > > > If I understand correctly you want both log4j-core main classes and test > > classes be distributed as modularized jars. > > To me with JPMS you should move the modularized test-classes to a separate > > Maven module as a first citizen artifact. > > That should make it a lot easier. > > It should also make the separation more clear: classes under > > target/test-classes are there to unittest its main classes. > > Whereas this new module contains reusable (main) code for testing (which > > should be tested too ;) ) > > I understand that this will change the GA(-classifier) for the testing > > module, but to me for a good reason. > > > > Even with your very small reduction of steps to simplify the process, it > > doesn't fit in the default lifecycle unless you write you introduce a new > > lifecycle-binding or write a new extension. > > > > Robert > > > > On 22-6-2021 07:02:59, Ralph Goers wrote: > > Sorry for posting again. I really need to proof-read better. Please ignore > > the prior email. > > > > I have recently had quite an adventure modifying several of Log4j’s Maven > > modules to > > implement JPMS on our master branch. It was an adventure due to a few > > issues: > > > > 1. https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826 . > > This bug has been fixed in Java 17 but Log4j uses Java 11 to build. > > 2. Log4j-plugins builds an annotation processor, packages it with the > > annotations > > and classes necessary to build and run plugins, creates and test jar and > > runs unit tests. > > 3. It is not possible to compile an annotation processor with a > > module-info.java present. > > The compile will fail because it can’t find the annotation processor > > “service” when > > compiling module-info.java. > > 4. It is very difficult to compile an annotation processor and then use it > > in the same Maven > > module. JPMS expects the annotation processor to either be on the classpath > > or specified > > with the processorpath option. When a module-info.java is present, Maven > > automatically > > moves everything to the module path. Maven only supports using coordinates > > to specify the > > processor path, which don’t exist when the processor is in the same module. > > The only way > > to solve this is to compile all the classes with proc=only without a > > module-info.java present. > > 5. Although number 4 might seem bad, it really doesn’t matter because javac > > will fail if a > > module-info.java is present because module-info.java will have a reference > > to the service > > class being generated by the annotation processor and for some reason > > module-info.java > > is resolved before the annotation processor runs. > > 6. If the main set of
Re: JPMS, Maven and test jars
Yes, you understand correctly. And yes, the test classes could be moved to a separate maven module, but then the unit tests would also have to move as well as they are dependent on those test classes. Of course, the test classes are dependent on the main source. As an example, log4j-core has a LoggerContext class, the test jar has a LoggerContextRule class that uses the LoggerContext and the LoggerContextRule is used by many unit tests. So now you would have a maven module with only the main source and no tests, and a second module where the “main” source is actually the test classes and the unit tests test the code in a different Maven module with a module info that opens the module-info of the main source jar. Seems downright weird to me. Yes, I realize this doesn’t fit in the normal Maven lifecycle. I brought it up because if and/or when more stuff starts becoming JPMS modularized this issue is going to become more and more common. To be honest, I’d be OK living with the build the way it is except people are complaining that their IDE has no idea what to do with the project. Ralph > On Jun 22, 2021, at 11:16 AM, Robert Scholte wrote: > > If I understand correctly you want both log4j-core main classes and test > classes be distributed as modularized jars. > To me with JPMS you should move the modularized test-classes to a separate > Maven module as a first citizen artifact. > That should make it a lot easier. > It should also make the separation more clear: classes under > target/test-classes are there to unittest its main classes. > Whereas this new module contains reusable (main) code for testing (which > should be tested too ;) ) > I understand that this will change the GA(-classifier) for the testing > module, but to me for a good reason. > > Even with your very small reduction of steps to simplify the process, it > doesn't fit in the default lifecycle unless you write you introduce a new > lifecycle-binding or write a new extension. > > Robert > > On 22-6-2021 07:02:59, Ralph Goers wrote: > Sorry for posting again. I really need to proof-read better. Please ignore > the prior email. > > I have recently had quite an adventure modifying several of Log4j’s Maven > modules to > implement JPMS on our master branch. It was an adventure due to a few issues: > > 1. https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826 . > This bug has been fixed in Java 17 but Log4j uses Java 11 to build. > 2. Log4j-plugins builds an annotation processor, packages it with the > annotations > and classes necessary to build and run plugins, creates and test jar and runs > unit tests. > 3. It is not possible to compile an annotation processor with a > module-info.java present. > The compile will fail because it can’t find the annotation processor > “service” when > compiling module-info.java. > 4. It is very difficult to compile an annotation processor and then use it in > the same Maven > module. JPMS expects the annotation processor to either be on the classpath > or specified > with the processorpath option. When a module-info.java is present, Maven > automatically > moves everything to the module path. Maven only supports using coordinates to > specify the > processor path, which don’t exist when the processor is in the same module. > The only way > to solve this is to compile all the classes with proc=only without a > module-info.java present. > 5. Although number 4 might seem bad, it really doesn’t matter because javac > will fail if a > module-info.java is present because module-info.java will have a reference to > the service > class being generated by the annotation processor and for some reason > module-info.java > is resolved before the annotation processor runs. > 6. If the main set of classes are compiled with a module-info.java every > other compile > in the Maven module must also be modularized. Likewise, if the main module > does > not contain a module-info.java no other compile can either. > 7. JPMS requires that every module have its own package space with no overlap > with any other JPMS module. > > So while generating the log4j-plugins module is quite painful, generating > log4j-core isn’t > much better. That is actually the primary focus for this list. > > Log4j-core consists of the main classes packaged in the jar, a test jar that > is used by > downstream Maven modules, and the unit tests. Prior to JPMS one would just > create > the main jar and then package all the test classes and unit tests in a test > jar. This can > no longer be done with JPMS. > > When a project publishes a test jar along with the main jar, just like any > other JPMS > module. the test jar cannot use the package space of the main classes. Log4j > core > uses org.apache.logging.log4j.core so I placed all the test utility classes > under > org.apache.logging.log4j.core.test. However, the unit tests all need to be > packaged > in the main package space since
Re: JPMS, Maven and test jars
If I understand correctly you want both log4j-core main classes and test classes be distributed as modularized jars. To me with JPMS you should move the modularized test-classes to a separate Maven module as a first citizen artifact. That should make it a lot easier. It should also make the separation more clear: classes under target/test-classes are there to unittest its main classes. Whereas this new module contains reusable (main) code for testing (which should be tested too ;) ) I understand that this will change the GA(-classifier) for the testing module, but to me for a good reason. Even with your very small reduction of steps to simplify the process, it doesn't fit in the default lifecycle unless you write you introduce a new lifecycle-binding or write a new extension. Robert On 22-6-2021 07:02:59, Ralph Goers wrote: Sorry for posting again. I really need to proof-read better. Please ignore the prior email. I have recently had quite an adventure modifying several of Log4j’s Maven modules to implement JPMS on our master branch. It was an adventure due to a few issues: 1. https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826 . This bug has been fixed in Java 17 but Log4j uses Java 11 to build. 2. Log4j-plugins builds an annotation processor, packages it with the annotations and classes necessary to build and run plugins, creates and test jar and runs unit tests. 3. It is not possible to compile an annotation processor with a module-info.java present. The compile will fail because it can’t find the annotation processor “service” when compiling module-info.java. 4. It is very difficult to compile an annotation processor and then use it in the same Maven module. JPMS expects the annotation processor to either be on the classpath or specified with the processorpath option. When a module-info.java is present, Maven automatically moves everything to the module path. Maven only supports using coordinates to specify the processor path, which don’t exist when the processor is in the same module. The only way to solve this is to compile all the classes with proc=only without a module-info.java present. 5. Although number 4 might seem bad, it really doesn’t matter because javac will fail if a module-info.java is present because module-info.java will have a reference to the service class being generated by the annotation processor and for some reason module-info.java is resolved before the annotation processor runs. 6. If the main set of classes are compiled with a module-info.java every other compile in the Maven module must also be modularized. Likewise, if the main module does not contain a module-info.java no other compile can either. 7. JPMS requires that every module have its own package space with no overlap with any other JPMS module. So while generating the log4j-plugins module is quite painful, generating log4j-core isn’t much better. That is actually the primary focus for this list. Log4j-core consists of the main classes packaged in the jar, a test jar that is used by downstream Maven modules, and the unit tests. Prior to JPMS one would just create the main jar and then package all the test classes and unit tests in a test jar. This can no longer be done with JPMS. When a project publishes a test jar along with the main jar, just like any other JPMS module. the test jar cannot use the package space of the main classes. Log4j core uses org.apache.logging.log4j.core so I placed all the test utility classes under org.apache.logging.log4j.core.test. However, the unit tests all need to be packaged in the main package space since its module-info.java “extends” the main module and several unit tests are required to be in the same main package so that they can access package private methods specifically provided for testing. In order to get this to work I had to perform the following steps: • Compile all the main classes except module-info.java with the Plugin annotation processor. • Compile the main module-info.java. • Compile the test classes used by other modules with module-info.java and using the plugin preprocessor. • Package these test classes in a test jar. • Delete the module-info and generated source for the test classes. • Move the main module-info to a temp location. • Compile the unit test classes without module-info.java. • Move the main module-info back to the classes directory. • Compile module-info.java for unit tests. • Run the unit tests. • Create the main jar if the unit tests pass. Were it not for JDK-8265826 I believe this could have been simplified somewhat to: • Compile all the main classes except module-info.java with the Plugin annotation processor. • Compile the main module-info.java. • Compile the test classes used by other modules with its module-info.java and using the plugin preprocessor. • Package these test classes in a test jar. • Delete the module-info and generated source for the test classes. • Compile the unit
Re: [VOTE] Release Maven Doxia version 1.10
+1 On Mon, Jun 21, 2021, 22:23 Michael Osipov wrote: > Hi, > > We solved 9 issues: > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12317230=12346985 > > There are still a couple of issues left in JIRA: > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20DOXIA%20AND%20resolution%20%3D%20Unresolved > > Staging repo: > https://repository.apache.org/content/repositories/maven-1652/ > > https://repository.apache.org/content/repositories/maven-1652/org/apache/maven/doxia/doxia/1.10/doxia-1.10-source-release.zip > > Source release checksum(s): > doxia-1.10-source-release.zip > sha512: > > a4622610a846d5aa6b267c716e608aa3ae5664d2417e8072555e7ccb0f9289f77a094ec5051201c88d2330dc9d7593cfe1b3f2645eef41521bc1c8c97d4d0f25 > > Staging site: > http://maven.apache.org/doxia/doxia-archives/doxia-LATEST/ > > Guide to testing staged releases: > http://maven.apache.org/guides/development/guide-testing-releases.html > > Vote open for 72 hours. > > [ ] +1 > [ ] +0 > [ ] -1 > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > >
Re: JPMS, Maven and test jars
Have you tried using the ModiTect Maven plugin? https://github.com/moditect/moditect There are many ways to use it: - with explicit module-info.java - with an embedded DSL in the pom.xml - with an embedded module descriptor in the pom.xml Ii use the last option on Jipsy [https://github.com/kordamp/jipsy/], which happens to define an APT as well. https://github.com/kordamp/jipsy/blob/master/jipsy-processor/pom.xml Cheers, Andres --- Java Champion; Groovy Enthusiast http://andresalmiray.com http://www.linkedin.com/in/aalmiray -- What goes up, must come down. Ask any system administrator. There are 10 types of people in the world: Those who understand binary, and those who don't. To understand recursion, we must first understand recursion. On Tue, Jun 22, 2021 at 7:02 AM Ralph Goers wrote: > Sorry for posting again. I really need to proof-read better. Please ignore > the prior email. > > I have recently had quite an adventure modifying several of Log4j’s Maven > modules to > implement JPMS on our master branch. It was an adventure due to a few > issues: > > 1. https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826 < > https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826>. >This bug has been fixed in Java 17 but Log4j uses Java 11 to build. > 2. Log4j-plugins builds an annotation processor, packages it with the > annotations >and classes necessary to build and run plugins, creates and test jar > and runs unit tests. > 3. It is not possible to compile an annotation processor with a > module-info.java present. >The compile will fail because it can’t find the annotation processor > “service” when >compiling module-info.java. > 4. It is very difficult to compile an annotation processor and then use it > in the same Maven >module. JPMS expects the annotation processor to either be on the > classpath or specified >with the processorpath option. When a module-info.java is present, > Maven automatically >moves everything to the module path. Maven only supports using > coordinates to specify the >processor path, which don’t exist when the processor is in the same > module. The only way >to solve this is to compile all the classes with proc=only without a > module-info.java present. > 5. Although number 4 might seem bad, it really doesn’t matter because > javac will fail if a >module-info.java is present because module-info.java will have a > reference to the service >class being generated by the annotation processor and for some reason > module-info.java >is resolved before the annotation processor runs. > 6. If the main set of classes are compiled with a module-info.java every > other compile >in the Maven module must also be modularized. Likewise, if the main > module does >not contain a module-info.java no other compile can either. > 7. JPMS requires that every module have its own package space with no > overlap > with any other JPMS module. > > So while generating the log4j-plugins module is quite painful, generating > log4j-core isn’t > much better. That is actually the primary focus for this list. > > Log4j-core consists of the main classes packaged in the jar, a test jar > that is used by > downstream Maven modules, and the unit tests. Prior to JPMS one would just > create > the main jar and then package all the test classes and unit tests in a > test jar. This can > no longer be done with JPMS. > > When a project publishes a test jar along with the main jar, just like any > other JPMS > module. the test jar cannot use the package space of the main classes. > Log4j core > uses org.apache.logging.log4j.core so I placed all the test utility > classes under > org.apache.logging.log4j.core.test. However, the unit tests all need to be > packaged > in the main package space since its module-info.java “extends” the main > module and > several unit tests are required to be in the same main package so that > they can access > package private methods specifically provided for testing. > > In order to get this to work I had to perform the following steps: > > • Compile all the main classes except module-info.java with the > Plugin annotation processor. > • Compile the main module-info.java. > • Compile the test classes used by other modules with > module-info.java and > using the plugin preprocessor. > • Package these test classes in a test jar. > • Delete the module-info and generated source for the test classes. > • Move the main module-info to a temp location. > • Compile the unit test classes without module-info.java. > • Move the main module-info back to the classes directory. > • Compile module-info.java for unit tests. > • Run the unit tests. > • Create the main jar if the unit tests pass. > > Were it not for JDK-8265826 I believe this could have been simplified > somewhat to: > >
