Re: JUnit assertAll and assertThrows question
Hi. Le jeu. 15 oct. 2020 à 17:29, Alex Herbert a écrit : > > > > > On 15 Oct 2020, at 15:56, John Patrick wrote: > > > > <— snip --> > > > > Will look at vfs and maybe also rng regarding tidying up the tests etc. > > Did you mean ’tidying up math’? > > Regarding RNG, [...] I fully concur with Alex that [RNG] is the last thing to change. The ratio of covered functionality to lines of codes is probably one of the highest in this project, thanks to him. And it is a *feature* that one can actually see some tests fail sometimes, with the expected probability! :-) In Commons Math however, it is mildly frightening that some tests only pass for a small proportion of the possible seeds. At best, such unit tests seem ill defined... Regards, Gilles P.S. For [RNG], the only thing worth (IMHO) upgrading the minimum JDK is JPMS indeed (cf. "commons-rng-examples/examples-jpms" maven module). Thus: Java 9 or beyond. - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: JUnit assertAll and assertThrows question
It's been a minor struggle to get components to Java 8, despite my efforts. Java 11 is a ways off especially since not all platforms support it yet (IBM iSeries IIRC, which matters to me). I'm all for getting all of Commons to Java 8. Gary On Thu, Oct 15, 2020, 10:57 John Patrick wrote: > Circa 2016 or 2017 I was trying to get commons projects ready for java > 9, so trying to get Automatic-Module-Name added into the manifest. > Moved onto other non commons projects for 2018 and 2019, helped with > modules and jupiter upgrades. > End of 2019 most of the java 9 work on other projects was blocked as > everything had dependencies on commons projects before they could be > true java 9+ modules projects. > Attempt to suggest a commons roadmap towards Java 11 as min version, > i.e. release current code at current version, bump to 1.8 and release > by end of 2020 say. Bump to Java 11 and release by mid 2021. So for a > year or 2 some projects might have multiple active master branches, so > if people do need commons-X on jdk 1.6 it is supported, but if no one > reports any issues it's maybe closed at the end of 2022. > Start of this year, try sorting out missing .gitignore and getting > them more standard, as some have target/ others /target/ others > **/target others **/target/. Upgrade to hamcrest v2.x. Move from junit > v4.x to junit-vintage v5.x and add junit-jupiter so dual support. Move > tests to junit jupiter classes. Upgrade maven dependencies/plugins to > latest versions. Look at adding using toolchain to select the jdk > instead of using the environment version. > Adding mvnw in prep for maven 3.7.0 which will have it now as part of > core maven instead of takari mvnw. So the project controls the maven > version, and when to upgrade. > I was going to start suggesting looking at moving commons-* to > org.apache.commons groupId so all are standard. Also how commons-lang3 > does it's packages so commons-lang and also be a dependency and have > different packages so they don't clash. But held off... > > Hope that gives an insight into my history with commons... some are > emails, some are jira tickets conversions, some are github > conversations. > > My goal is to get commons upgrade to jpms, so the downstream projects > I help out in 2018/2019 can upgrade and do a final jdk 1.8 release and > move onto jdk 11. > > So pr's for junit assertThrows I think are all done for > commons-collections, commons-digester and commons-validator. > > Will look at vfs and maybe also rng regarding tidying up the tests etc. > > John > > On Thu, 15 Oct 2020 at 13:06, Gilles Sadowski > wrote: > > > > Hello. > > > > Le mer. 14 oct. 2020 à 21:41, John Patrick a > écrit : > > > > > > before i waste time looking at upgrading tests... > > > > > > any objections if i upgrade tests to use assertAll and assertThrows > > > introduced in JUnit jupiter? > > > > > > I see it as less tech debt removal and I'm happy to spend time doing > > > the upgrade which I've done from maybe several projects now. Just > > > don't want to get to raising PR and them being rejected, which I feel > > > happens with everything I try to help out with related to commons > > > projects at the moment... > > > > I did notice that (or forgot it). Could you please remind of what you've > > proposed? > > > > AFAICT, the main contributors to > > Commons Math > > Commons Numbers > > Commons Statistics > > Commons Geometry > > Commons RNG > > are glad to keep in line with the evolution of the best Java practices. > > Except for [RNG], which requires Java 1.6, all the above components > > are at Java 8. > > > > If you are into improving our testing suites, please note that help > > would be welcome in ironing out issues in Commons Math, where > > sometimes (but too often) the use of "randomly" generated data > > with a *fixed seed* hides the fact that the test actually fails for most > > input (i.e. data generated from another seed). > > > > > > Thanks, > > Gilles > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > > For additional commands, e-mail: dev-h...@commons.apache.org > > > > - > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > >
Re: JUnit assertAll and assertThrows question
> On 15 Oct 2020, at 15:56, John Patrick wrote: > > <— snip --> > > Will look at vfs and maybe also rng regarding tidying up the tests etc. Did you mean ’tidying up math’? Regarding RNG, the test suite relies on the fact that statistical tests can fail and be rerun. This uses a feature in JUnit 4 that is applied in the config for the surefire plugin: 2 It applies to the modules: commons-rng-core commons-rng-sampling commons-rng-simple As far as I know there is no equivalent in JUnit 5. The p-values for most tests are set at 1/100. With 2 reruns the fail rate for single tests is 1/10. If I recall correctly when I looked at this the overall fail rate for the rng-core test suite is around 1/600. I am not sure about the others but there are fewer tests in those and the reruns is set at 1 and not 2. The failures are expected. This is a random sampling library. It cannot be tested with fixed seeds as then it is not fully exercising the randomness or the self-seeding functionality. The result is that any failures on the CI system just have to be looked at across the different Java builds. If the failures are in the same place it should be looked at, otherwise the job can be restarted and should pass second time. Currently it is more helpful to leave the test suite as is rather than change the p-value to 1/10 for tests which is what would be required on JUnit 5 without reruns. Without change to the test suite the back log of CI results can actually be used to analyse the failure statistics (although I’ve not done this non-locally). So until JUnit 5 offers a reruns facility the easiest action is to remain at JUnit 4. Any thoughts on this with regard to an upgrade path are welcome. Alex
Re: JUnit assertAll and assertThrows question
Circa 2016 or 2017 I was trying to get commons projects ready for java 9, so trying to get Automatic-Module-Name added into the manifest. Moved onto other non commons projects for 2018 and 2019, helped with modules and jupiter upgrades. End of 2019 most of the java 9 work on other projects was blocked as everything had dependencies on commons projects before they could be true java 9+ modules projects. Attempt to suggest a commons roadmap towards Java 11 as min version, i.e. release current code at current version, bump to 1.8 and release by end of 2020 say. Bump to Java 11 and release by mid 2021. So for a year or 2 some projects might have multiple active master branches, so if people do need commons-X on jdk 1.6 it is supported, but if no one reports any issues it's maybe closed at the end of 2022. Start of this year, try sorting out missing .gitignore and getting them more standard, as some have target/ others /target/ others **/target others **/target/. Upgrade to hamcrest v2.x. Move from junit v4.x to junit-vintage v5.x and add junit-jupiter so dual support. Move tests to junit jupiter classes. Upgrade maven dependencies/plugins to latest versions. Look at adding using toolchain to select the jdk instead of using the environment version. Adding mvnw in prep for maven 3.7.0 which will have it now as part of core maven instead of takari mvnw. So the project controls the maven version, and when to upgrade. I was going to start suggesting looking at moving commons-* to org.apache.commons groupId so all are standard. Also how commons-lang3 does it's packages so commons-lang and also be a dependency and have different packages so they don't clash. But held off... Hope that gives an insight into my history with commons... some are emails, some are jira tickets conversions, some are github conversations. My goal is to get commons upgrade to jpms, so the downstream projects I help out in 2018/2019 can upgrade and do a final jdk 1.8 release and move onto jdk 11. So pr's for junit assertThrows I think are all done for commons-collections, commons-digester and commons-validator. Will look at vfs and maybe also rng regarding tidying up the tests etc. John On Thu, 15 Oct 2020 at 13:06, Gilles Sadowski wrote: > > Hello. > > Le mer. 14 oct. 2020 à 21:41, John Patrick a écrit : > > > > before i waste time looking at upgrading tests... > > > > any objections if i upgrade tests to use assertAll and assertThrows > > introduced in JUnit jupiter? > > > > I see it as less tech debt removal and I'm happy to spend time doing > > the upgrade which I've done from maybe several projects now. Just > > don't want to get to raising PR and them being rejected, which I feel > > happens with everything I try to help out with related to commons > > projects at the moment... > > I did notice that (or forgot it). Could you please remind of what you've > proposed? > > AFAICT, the main contributors to > Commons Math > Commons Numbers > Commons Statistics > Commons Geometry > Commons RNG > are glad to keep in line with the evolution of the best Java practices. > Except for [RNG], which requires Java 1.6, all the above components > are at Java 8. > > If you are into improving our testing suites, please note that help > would be welcome in ironing out issues in Commons Math, where > sometimes (but too often) the use of "randomly" generated data > with a *fixed seed* hides the fact that the test actually fails for most > input (i.e. data generated from another seed). > > > Thanks, > Gilles > > - > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: JUnit assertAll and assertThrows question
Hello. Le mer. 14 oct. 2020 à 21:41, John Patrick a écrit : > > before i waste time looking at upgrading tests... > > any objections if i upgrade tests to use assertAll and assertThrows > introduced in JUnit jupiter? > > I see it as less tech debt removal and I'm happy to spend time doing > the upgrade which I've done from maybe several projects now. Just > don't want to get to raising PR and them being rejected, which I feel > happens with everything I try to help out with related to commons > projects at the moment... I did notice that (or forgot it). Could you please remind of what you've proposed? AFAICT, the main contributors to Commons Math Commons Numbers Commons Statistics Commons Geometry Commons RNG are glad to keep in line with the evolution of the best Java practices. Except for [RNG], which requires Java 1.6, all the above components are at Java 8. If you are into improving our testing suites, please note that help would be welcome in ironing out issues in Commons Math, where sometimes (but too often) the use of "randomly" generated data with a *fixed seed* hides the fact that the test actually fails for most input (i.e. data generated from another seed). Thanks, Gilles - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: JUnit assertAll and assertThrows question
Looking at commons-collections, commons-digester and commons-validator
at the moment.
Will look at commons-vfs next regarding getting to JUnit v4, then
adding assertThrows afterwards.
What are views about adding
`1.8`? I've not
done it at the moment as suspect builds are not setup for that and
will fail etc.
But it does mean I need to add;
final ThrowingRunnable testMethod = new ThrowingRunnable() {
public void run() {
// The Code
}
};
assertThrows(ExceptionClass.class, testMethod);
instead of just;
final Executable testMethod = () -> // The Code
assertThrows(ExceptionClass.class, testMethod);
John
On Wed, 14 Oct 2020 at 22:07, Gary Gregory wrote:
>
> For my money:
>
> - I REALLY like assertThrows
> - Might as well go to JUnit 5 if you feel up for it.
> - BUT: My biggest pain point ATM is that while Commons VFS is on JUnit 4,
> most tests are still JUnit 3 (TestCase) classes that won't run properly
> from Eclipse.
>
> So I would really like help getting VFS to at least be off of anything
> JUnit 3-like and at least to JUnit 4 but that won't stop a release which I
> hope to do within a week or two.
>
> TY!
> Gary
>
> On Wed, Oct 14, 2020 at 3:56 PM Romain Manni-Bucau
> wrote:
>
> > Hi John,
> >
> > Would be +1 from me if:
> >
> > 1. It does not require to move from junit4 to junit5 (no gain IMHO)
> > 2. Test goes further than a Exception rule or @Test(expected)
> >
> > If both points are matched it makes sense to me, otherwise it will likely
> > add more issues from my experience.
> >
> >
> > Le mer. 14 oct. 2020 à 21:40, John Patrick a
> > écrit :
> >
> > > before i waste time looking at upgrading tests...
> > >
> > > any objections if i upgrade tests to use assertAll and assertThrows
> > > introduced in JUnit jupiter?
> > >
> > > I see it as less tech debt removal and I'm happy to spend time doing
> > > the upgrade which I've done from maybe several projects now. Just
> > > don't want to get to raising PR and them being rejected, which I feel
> > > happens with everything I try to help out with related to commons
> > > projects at the moment...
> > >
> > > cheers,
> > > john
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > > For additional commands, e-mail: dev-h...@commons.apache.org
> > >
> > >
> >
-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org
Re: JUnit assertAll and assertThrows question
For my money: - I REALLY like assertThrows - Might as well go to JUnit 5 if you feel up for it. - BUT: My biggest pain point ATM is that while Commons VFS is on JUnit 4, most tests are still JUnit 3 (TestCase) classes that won't run properly from Eclipse. So I would really like help getting VFS to at least be off of anything JUnit 3-like and at least to JUnit 4 but that won't stop a release which I hope to do within a week or two. TY! Gary On Wed, Oct 14, 2020 at 3:56 PM Romain Manni-Bucau wrote: > Hi John, > > Would be +1 from me if: > > 1. It does not require to move from junit4 to junit5 (no gain IMHO) > 2. Test goes further than a Exception rule or @Test(expected) > > If both points are matched it makes sense to me, otherwise it will likely > add more issues from my experience. > > > Le mer. 14 oct. 2020 à 21:40, John Patrick a > écrit : > > > before i waste time looking at upgrading tests... > > > > any objections if i upgrade tests to use assertAll and assertThrows > > introduced in JUnit jupiter? > > > > I see it as less tech debt removal and I'm happy to spend time doing > > the upgrade which I've done from maybe several projects now. Just > > don't want to get to raising PR and them being rejected, which I feel > > happens with everything I try to help out with related to commons > > projects at the moment... > > > > cheers, > > john > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > > For additional commands, e-mail: dev-h...@commons.apache.org > > > > >
Re: JUnit assertAll and assertThrows question
Hi John, Would be +1 from me if: 1. It does not require to move from junit4 to junit5 (no gain IMHO) 2. Test goes further than a Exception rule or @Test(expected) If both points are matched it makes sense to me, otherwise it will likely add more issues from my experience. Le mer. 14 oct. 2020 à 21:40, John Patrick a écrit : > before i waste time looking at upgrading tests... > > any objections if i upgrade tests to use assertAll and assertThrows > introduced in JUnit jupiter? > > I see it as less tech debt removal and I'm happy to spend time doing > the upgrade which I've done from maybe several projects now. Just > don't want to get to raising PR and them being rejected, which I feel > happens with everything I try to help out with related to commons > projects at the moment... > > cheers, > john > > - > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > >
JUnit assertAll and assertThrows question
before i waste time looking at upgrading tests... any objections if i upgrade tests to use assertAll and assertThrows introduced in JUnit jupiter? I see it as less tech debt removal and I'm happy to spend time doing the upgrade which I've done from maybe several projects now. Just don't want to get to raising PR and them being rejected, which I feel happens with everything I try to help out with related to commons projects at the moment... cheers, john - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
