Re: [DISCUSS] JUnit 5 Customization

[Aleksei Zotov]

Hi Jaikiran,

Were you able to check the PRs? I'd would be great to get into the next
release with these changes (or alternative workarounds), so we (Apache
Cassandra) can move forward with JUnit 5 upgrade.


Best Regards,

Aleksei Zotov.


On Thu, Dec 23, 2021 at 5:22 PM Aleksei Zotov  wrote:

> Jaikiran,
>
> Thank you for the quick response!
>
> ant -f fetch.xml -Ddest=optional
>
> It helped, thanks for pointing this out!
>
> I can understand that some of the code in the AbstractJUnitResultFormatter
>> might appear reusable and would be tempting to reuse/extend, but I wouldn't
>> want anything outside of Ant to start using these classes and instead just
>> code against the JUnit5 classes/interfaces.
>
> Having said all this, if any of other maintainers of the Ant project feel
>> that we should allow these internal classes to be extensible, I won't mind
>> having this work reviewed and merged.
>
> I truly understand your point of view. If no-one else needs this
> functionality we'll simply implement custom formatters/listeners based on
> the existing implementation in *ant* project.
>
> I'll take a look at this one.
>
> Great, thanks.
>
>
> Best Regards,
>
> Aleksei Zotov.
>
>
> On Thu, Dec 23, 2021 at 6:34 AM Jaikiran Pai 
> wrote:
>
>> Hello Aleksei,
>>
>> On 22/12/21 7:59 pm, Aleksei Zotov wrote:
>> > Hi Ant Developers,
>> >
>> > I'm working on the migration of Apache Cassandra project from JUnit 4 to
>> > JUnit 5 (CASSANDRA-16630
>> > ) which turned
>> out
>> > to be larger than I originally expected. At the moment, three PRs got
>> > merged:
>> >
>> > - https://github.com/apache/ant/pull/147
>> > - https://github.com/apache/ant/pull/168
>> > - https://github.com/apache/ant/pull/172
>> >
>> > And there are a few more I'd like to discuss with you.
>>
>> Sorry, I remember seeing a PR from you where you asked for some inputs
>> in this area, but I haven't been able to get to it due to some other
>> priorities.
>>
>> >
>> > Formatters Extensibility
>> > Apache Cassandra extended the existing JUnit 4 formatters in order to
>> > integrate them with a custom logic required for a better CI process. It
>> was
>> > easily achievable for JUnit 4 since the formatters are marked public. On
>> > the contrary, for JUnit 5 all formatters are package private. Even
>> > *AbstractJUnitResultFormatter* is package private. Of course, we can
>> > copy-paste all these classes to our project and customize them based on
>> our
>> > needs, but that seems to be a bit of an overhead.
>> >
>> > Currently, we'd like to make *AbstractJUnitResultFormatter *extensible.
>> > Here is the corresponding PR: https://github.com/apache/ant/pull/169/
>>
>> The reason why these formatters are package private and not extensible
>> is intentional. When I added these formatters, one of the goals of these
>> formatters was to only make them generate output to an extent that it
>> matches the JUnit4 style formatters of the "junit" task (hence the name
>> "legacy-" to those formatters). All these pre-shipped "legacy-"
>> listeners that you see in Ant are only there for minimal support and
>> their implementation is considered to be internal to the Ant distribution.
>>
>> The new JUnit5 framework itself is extensible and also comes with some
>> pre-shipped "listeners" (implementations of TestExecutionListener).
>> That's one of the reasons why the junitlauncher task's "listener"
>> element provides a direct way to use any implementation of the JUnit5
>> framework's "org.junit.platform.launcher.TestExecutionListener". i.e.
>> for any code that seeks extensibility, the
>> org.junit.platform.launcher.TestExecutionListener is expected to be the
>> extension point/interface and not Ant's internal
>> AbstractJUnitResultFormatter. This allows the junitlauncher task to be
>> minimal and allows it to achieve its goal of just being something that
>> will launch the JUnit5 framework.
>>
>> I can understand that some of the code in the
>> AbstractJUnitResultFormatter might appear reusable and would be tempting
>> to reuse/extend, but I wouldn't want anything outside of Ant to start
>> using these classes and instead just code against the JUnit5
>> classes/interfaces.
>>
>> Having said all this, if any of other maintainers of the Ant project
>> feel that we should allow these internal classes to be extensible, I
>> won't mind having this work reviewed and merged.
>>
>> > Fork Mode Support
>> > Apache Cassandra is a large and complex product and in order to
>> guarantee
>> > its quality we run many tests independently. It lets us ensure different
>> > test suites do not affect each other. For isolated testing we spin up a
>> new
>> > JVM per tests suite via *forkMode* property. Unfortunately,
>> > *junitlauncher* task
>> > does not provide such a functionality.
>> >
>> > Currently, we'd like *mode* attribute to *fork* element. Here is the
>> > corresponding PR: 

Re: [DISCUSS] JUnit 5 Customization

[Aleksei Zotov]

Jaikiran,

Thank you for the quick response!

ant -f fetch.xml -Ddest=optional

It helped, thanks for pointing this out!

I can understand that some of the code in the AbstractJUnitResultFormatter
> might appear reusable and would be tempting to reuse/extend, but I wouldn't
> want anything outside of Ant to start using these classes and instead just
> code against the JUnit5 classes/interfaces.

Having said all this, if any of other maintainers of the Ant project feel
> that we should allow these internal classes to be extensible, I won't mind
> having this work reviewed and merged.

I truly understand your point of view. If no-one else needs this
functionality we'll simply implement custom formatters/listeners based on
the existing implementation in *ant* project.

I'll take a look at this one.

Great, thanks.


Best Regards,

Aleksei Zotov.


On Thu, Dec 23, 2021 at 6:34 AM Jaikiran Pai 
wrote:

> Hello Aleksei,
>
> On 22/12/21 7:59 pm, Aleksei Zotov wrote:
> > Hi Ant Developers,
> >
> > I'm working on the migration of Apache Cassandra project from JUnit 4 to
> > JUnit 5 (CASSANDRA-16630
> > ) which turned
> out
> > to be larger than I originally expected. At the moment, three PRs got
> > merged:
> >
> > - https://github.com/apache/ant/pull/147
> > - https://github.com/apache/ant/pull/168
> > - https://github.com/apache/ant/pull/172
> >
> > And there are a few more I'd like to discuss with you.
>
> Sorry, I remember seeing a PR from you where you asked for some inputs
> in this area, but I haven't been able to get to it due to some other
> priorities.
>
> >
> > Formatters Extensibility
> > Apache Cassandra extended the existing JUnit 4 formatters in order to
> > integrate them with a custom logic required for a better CI process. It
> was
> > easily achievable for JUnit 4 since the formatters are marked public. On
> > the contrary, for JUnit 5 all formatters are package private. Even
> > *AbstractJUnitResultFormatter* is package private. Of course, we can
> > copy-paste all these classes to our project and customize them based on
> our
> > needs, but that seems to be a bit of an overhead.
> >
> > Currently, we'd like to make *AbstractJUnitResultFormatter *extensible.
> > Here is the corresponding PR: https://github.com/apache/ant/pull/169/
>
> The reason why these formatters are package private and not extensible
> is intentional. When I added these formatters, one of the goals of these
> formatters was to only make them generate output to an extent that it
> matches the JUnit4 style formatters of the "junit" task (hence the name
> "legacy-" to those formatters). All these pre-shipped "legacy-"
> listeners that you see in Ant are only there for minimal support and
> their implementation is considered to be internal to the Ant distribution.
>
> The new JUnit5 framework itself is extensible and also comes with some
> pre-shipped "listeners" (implementations of TestExecutionListener).
> That's one of the reasons why the junitlauncher task's "listener"
> element provides a direct way to use any implementation of the JUnit5
> framework's "org.junit.platform.launcher.TestExecutionListener". i.e.
> for any code that seeks extensibility, the
> org.junit.platform.launcher.TestExecutionListener is expected to be the
> extension point/interface and not Ant's internal
> AbstractJUnitResultFormatter. This allows the junitlauncher task to be
> minimal and allows it to achieve its goal of just being something that
> will launch the JUnit5 framework.
>
> I can understand that some of the code in the
> AbstractJUnitResultFormatter might appear reusable and would be tempting
> to reuse/extend, but I wouldn't want anything outside of Ant to start
> using these classes and instead just code against the JUnit5
> classes/interfaces.
>
> Having said all this, if any of other maintainers of the Ant project
> feel that we should allow these internal classes to be extensible, I
> won't mind having this work reviewed and merged.
>
> > Fork Mode Support
> > Apache Cassandra is a large and complex product and in order to guarantee
> > its quality we run many tests independently. It lets us ensure different
> > test suites do not affect each other. For isolated testing we spin up a
> new
> > JVM per tests suite via *forkMode* property. Unfortunately,
> > *junitlauncher* task
> > does not provide such a functionality.
> >
> > Currently, we'd like *mode* attribute to *fork* element. Here is the
> > corresponding PR: https://github.com/apache/ant/pull/174
>
> I remember there was some bugzilla discussion of a similar question (or
> maybe on some other forum). I'll take a look at this PR soon.
>
> > I'd be glad if you could provide some feedback on that. Also I need some
> > guidance here - I have a suspicion that *JUnitLauncherTaskTest* is not
> > being run during "./build clean test" (I cannot grep it in logs). Could
> you
> > please help me to run this particular test 

Re: [DISCUSS] JUnit 5 Customization

[Jaikiran Pai]

Hello Aleksei,

On 22/12/21 7:59 pm, Aleksei Zotov wrote:

Hi Ant Developers,

I'm working on the migration of Apache Cassandra project from JUnit 4 to
JUnit 5 (CASSANDRA-16630
) which turned out
to be larger than I originally expected. At the moment, three PRs got
merged:

- https://github.com/apache/ant/pull/147
- https://github.com/apache/ant/pull/168
- https://github.com/apache/ant/pull/172

And there are a few more I'd like to discuss with you.


Sorry, I remember seeing a PR from you where you asked for some inputs 
in this area, but I haven't been able to get to it due to some other 
priorities.




Formatters Extensibility
Apache Cassandra extended the existing JUnit 4 formatters in order to
integrate them with a custom logic required for a better CI process. It was
easily achievable for JUnit 4 since the formatters are marked public. On
the contrary, for JUnit 5 all formatters are package private. Even
*AbstractJUnitResultFormatter* is package private. Of course, we can
copy-paste all these classes to our project and customize them based on our
needs, but that seems to be a bit of an overhead.

Currently, we'd like to make *AbstractJUnitResultFormatter *extensible.
Here is the corresponding PR: https://github.com/apache/ant/pull/169/


The reason why these formatters are package private and not extensible 
is intentional. When I added these formatters, one of the goals of these 
formatters was to only make them generate output to an extent that it 
matches the JUnit4 style formatters of the "junit" task (hence the name 
"legacy-" to those formatters). All these pre-shipped "legacy-" 
listeners that you see in Ant are only there for minimal support and 
their implementation is considered to be internal to the Ant distribution.


The new JUnit5 framework itself is extensible and also comes with some 
pre-shipped "listeners" (implementations of TestExecutionListener). 
That's one of the reasons why the junitlauncher task's "listener" 
element provides a direct way to use any implementation of the JUnit5 
framework's "org.junit.platform.launcher.TestExecutionListener". i.e. 
for any code that seeks extensibility, the 
org.junit.platform.launcher.TestExecutionListener is expected to be the 
extension point/interface and not Ant's internal 
AbstractJUnitResultFormatter. This allows the junitlauncher task to be 
minimal and allows it to achieve its goal of just being something that 
will launch the JUnit5 framework.


I can understand that some of the code in the 
AbstractJUnitResultFormatter might appear reusable and would be tempting 
to reuse/extend, but I wouldn't want anything outside of Ant to start 
using these classes and instead just code against the JUnit5 
classes/interfaces.


Having said all this, if any of other maintainers of the Ant project 
feel that we should allow these internal classes to be extensible, I 
won't mind having this work reviewed and merged.



Fork Mode Support
Apache Cassandra is a large and complex product and in order to guarantee
its quality we run many tests independently. It lets us ensure different
test suites do not affect each other. For isolated testing we spin up a new
JVM per tests suite via *forkMode* property. Unfortunately,
*junitlauncher* task
does not provide such a functionality.

Currently, we'd like *mode* attribute to *fork* element. Here is the
corresponding PR: https://github.com/apache/ant/pull/174


I remember there was some bugzilla discussion of a similar question (or 
maybe on some other forum). I'll take a look at this PR soon.



I'd be glad if you could provide some feedback on that. Also I need some
guidance here - I have a suspicion that *JUnitLauncherTaskTest* is not
being run during "./build clean test" (I cannot grep it in logs). Could you
please help me to run this particular test from CLI.


Have you fetched the optional JUnit5 libraries before running these 
tests? A lot of Ant's tasks are optional and their tests are only run if 
the corresponding libraries are available in the classpath. To fetch 
these dependencies you can do:


ant -f fetch.xml -Ddest=optional

and then run the tests using the command you currently use.


Use File Support
Apache Cassandra does not always need to write testing output to a file.
Even though it is possible to achieve writing into standard output (by
overriding *TestResultFormatter.setDestination *method) instead of a file,
it is impossible to prevent empty files from being created on the file
system. Of course, we can delete these files, however, it would be nice to
have the functionality similar to *junit* task.

I've already started a related discussion on the PR (please, chime in):
https://github.com/apache/ant/pull/171


I'll take a look at this one.

-Jaikiran


-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: 

[DISCUSS] JUnit 5 Customization

[Aleksei Zotov]

Hi Ant Developers,

I'm working on the migration of Apache Cassandra project from JUnit 4 to
JUnit 5 (CASSANDRA-16630
) which turned out
to be larger than I originally expected. At the moment, three PRs got
merged:

   - https://github.com/apache/ant/pull/147
   - https://github.com/apache/ant/pull/168
   - https://github.com/apache/ant/pull/172

And there are a few more I'd like to discuss with you.

Formatters Extensibility
Apache Cassandra extended the existing JUnit 4 formatters in order to
integrate them with a custom logic required for a better CI process. It was
easily achievable for JUnit 4 since the formatters are marked public. On
the contrary, for JUnit 5 all formatters are package private. Even
*AbstractJUnitResultFormatter* is package private. Of course, we can
copy-paste all these classes to our project and customize them based on our
needs, but that seems to be a bit of an overhead.

Currently, we'd like to make *AbstractJUnitResultFormatter *extensible.
Here is the corresponding PR: https://github.com/apache/ant/pull/169/

I'd be happy to make other existing formatters extensible as well. Please,
let me know if that sounds good to you and I'll add the changes to the
existing PR or raise another one.

Fork Mode Support
Apache Cassandra is a large and complex product and in order to guarantee
its quality we run many tests independently. It lets us ensure different
test suites do not affect each other. For isolated testing we spin up a new
JVM per tests suite via *forkMode* property. Unfortunately,
*junitlauncher* task
does not provide such a functionality.

Currently, we'd like *mode* attribute to *fork* element. Here is the
corresponding PR: https://github.com/apache/ant/pull/174

I'd be glad if you could provide some feedback on that. Also I need some
guidance here - I have a suspicion that *JUnitLauncherTaskTest* is not
being run during "./build clean test" (I cannot grep it in logs). Could you
please help me to run this particular test from CLI.

Use File Support
Apache Cassandra does not always need to write testing output to a file.
Even though it is possible to achieve writing into standard output (by
overriding *TestResultFormatter.setDestination *method) instead of a file,
it is impossible to prevent empty files from being created on the file
system. Of course, we can delete these files, however, it would be nice to
have the functionality similar to *junit* task.

I've already started a related discussion on the PR (please, chime in):
https://github.com/apache/ant/pull/171


I'd be very grateful to get code review comments on the PRs and your
feedback on the above points.


Best Regards,

Aleksei Zotov.