Re: Can the automated tests be included in the source tarball?
Hi Ole!
Please acknowledge you have received this message.
On Thu, 1 Oct 2020 20:22:46 +0300
Shlomi Fish wrote:
> Hi Ole!
>
> While trying to get the test suite (or its "Portable.sh" subset) to pass on
> Fedora 32 x86-64, I ran into these grammar errors:
>
> https://www.shlomifish.org/Files/files/code/parallel-mak-grammar-v0.1.0.patch
>
> Patch is under CC0 / MITL/ExpatL - I didn't see an obvious way to submit
> patches or merge reqs on
> https://savannah.gnu.org/bugs/?group=parallel=additem .
>
> Otherwise note that what I tried ran "sudo" and asked for my password. :(
>
> > [snipped]
>
The corrections in the patch have not been applied yet:
«
[shlomif@localhost parallel]$ git show | head
commit e1a62d362d869822941b14a7d00fb0c59bff3109
Author: Ole Tange
Date: Sun Nov 22 16:22:54 2020 +0100
Released as 20201122 ('Biden')
diff --git a/NEWS b/NEWS
index eb4ad0dd..f1f89e40 100644
--- a/NEWS
+++ b/NEWS
»
I've been deadlocked waiting for this (while working on other projects), and
it's been over two months.
Why wasn't it applied?
--
Shlomi Fish https://www.shlomifish.org/
https://www.shlomifish.org/humour/Summerschool-at-the-NSA/
I’d rather have natural stupidity instead of artificial intelligence.
— Unknown
Please reply to list if it's a mailing list post - https://shlom.in/reply .
Re: Can the automated tests be included in the source tarball?
On Thu, Oct 1, 2020 at 5:09 PM Shlomi Fish wrote: > On Tue, 29 Sep 2020 01:18:31 +0200 > Ole Tange wrote: > > > On Thu, Sep 24, 2020 at 8:37 AM Shlomi Fish wrote: : > > Unfortunately some of them are heavily dependent on running on the > > same equipment as mine due to timing issues (and every time I change > > hardware or upgrade software, I have to tweak the tests or results a > > little). > > I see. I suppose you are aware of running on a pristine CI system, e.g: I had a talk with a guy who used Jenkins. Jenkins is all good and fine for tests that have predictable outcomes no matter the environment. It is worse when you have 2 tests: One has to run while the machine is swapping and one has to run when the machine is _not_ swapping. In other words they cannot be run in parallel. Or: A test has to run on a machine with high CPU load, while another test has to run on an idle machine. Or: A test has to be run on Centos 3, because other distributions do not show the error. I guess around 20% of my tests are in that category. The rest could probably be run by a normal CI. The current home-made test system also has the advantage that if a test fails, it will be pushed to the top, so when I run the test suite again, it will be the first test to run. > > If you want to help, a good start would be to have > > testsuite/REQUIREMENTS ported so it would work on more systems. It > > (tries to) install and set up all required software to run the > > testsuite. > > Wow, it looks intimidating. And I thought some of my FOSS projects had too > many > prereqs! The problem is that errors are often seen in the interaction with other tools - sometimes only as race conditions. /Ole
Re: Can the automated tests be included in the source tarball?
Hi Ole! While trying to get the test suite (or its "Portable.sh" subset) to pass on Fedora 32 x86-64, I ran into these grammar errors: https://www.shlomifish.org/Files/files/code/parallel-mak-grammar-v0.1.0.patch Patch is under CC0 / MITL/ExpatL - I didn't see an obvious way to submit patches or merge reqs on https://savannah.gnu.org/bugs/?group=parallel=additem . Otherwise note that what I tried ran "sudo" and asked for my password. :( > [snipped] -- Shlomi Fish https://www.shlomifish.org/ Parody of "The Fountainhead" - https://shlom.in/towtf Mastering ‘cat’ is almost as difficult as herding cats. — https://www.shlomifish.org/humour/bits/Mastering-Cat/ Please reply to list if it's a mailing list post - https://shlom.in/reply . diff --git a/testsuite/Makefile b/testsuite/Makefile index 43572e7..847329e 100644 --- a/testsuite/Makefile +++ b/testsuite/Makefile @@ -7,7 +7,7 @@ testsuite: 3 touch ~/.parallel/will-cite make stopvm -### Limited testsets - run once +### Limited test sets - run once # No 100s, 300s, mem, polarhome, tutorial 1: ../src/parallel tests-to-run/* wanted-results/* prereqlocal startdb prereqremote TRIES=1 time bash Start.sh '' '00s|mem|polarhome|tutorial' || true @@ -58,7 +58,7 @@ testlocalshort: ../src/parallel tests-to-run/*local* wanted-results/*local* prer testdb: ../src/parallel tests-to-run/*sql* wanted-results/*sql* prereqdb time bash Start.sh sql NONE -# Portable tests that runs everywhere +# Portable tests that run everywhere portable: # Cache sudo password sudo echo
Re: Can the automated tests be included in the source tarball?
Hi Ole! Thanks for the reply. On Tue, 29 Sep 2020 01:18:31 +0200 Ole Tange wrote: > On Thu, Sep 24, 2020 at 8:37 AM Shlomi Fish wrote: > > > > Can the automated tests be included in the source tarball? It does not > > include them currently which makes assuring that it works properly harder. > > They are available in git. > > I am not sure they would make sense to normal users, thus I am not > convinced they should go into the tarball. > > Unfortunately some of them are heavily dependent on running on the > same equipment as mine due to timing issues (and every time I change > hardware or upgrade software, I have to tweak the tests or results a > little). > I see. I suppose you are aware of running on a pristine CI system, e.g: https://github.com/shlomif/Freenode-programming-channel-FAQ/blob/master/FAQ_with_ToC__generated.md#what-do-continuous-integration-ci-services-such-as-travis-ci-jenkins-or-appveyor-provide > But it would be a good idea to identify those that are not dependent > on this and make it clear that these should be able to run on all > systems. And if possible change more tests to be independent. Yes. > > If you want to help, a good start would be to have > testsuite/REQUIREMENTS ported so it would work on more systems. It > (tries to) install and set up all required software to run the > testsuite. Wow, it looks intimidating. And I thought some of my FOSS projects had too many prereqs! Of course, some users will still complain even if there is a moderate number of deps, which are even explicitly specified in the CI configuration (e.g.: https://github.com/shlomif/fortune-mod/issues/44 ). Anyway, one approach I used to avoid failing tests with certain configurations/missing-deps is to specify "tags" / "labels" in an environment variable and skip the test scripts (or individual assertions) if they are set or unset, e.g in: https://github.com/shlomif/fc-solve/blob/17426fe265a348fe1cd1df65d7fb1e1602711c86/fc-solve/source/t/t/fc-solve-output-Test-Trap.t#L207 (short url: https://is.gd/wyDrck ) > > GNU Parallel is pretty well tested on many systems. The testsuite is > not - not even close, so expect it to puke errors all over the place. > > > /Ole > Thanks! -- Shlomi Fish https://www.shlomifish.org/ Interview with Ben Collins-Sussman - https://shlom.in/sussman I’m trying to achieve world peace and this regex is the last thing standing in my way! ;) Please reply to list if it's a mailing list post - https://shlom.in/reply .
Re: Can the automated tests be included in the source tarball?
On Thu, Sep 24, 2020 at 8:37 AM Shlomi Fish wrote: > > Can the automated tests be included in the source tarball? It does not include > them currently which makes assuring that it works properly harder. They are available in git. I am not sure they would make sense to normal users, thus I am not convinced they should go into the tarball. Unfortunately some of them are heavily dependent on running on the same equipment as mine due to timing issues (and every time I change hardware or upgrade software, I have to tweak the tests or results a little). But it would be a good idea to identify those that are not dependent on this and make it clear that these should be able to run on all systems. And if possible change more tests to be independent. If you want to help, a good start would be to have testsuite/REQUIREMENTS ported so it would work on more systems. It (tries to) install and set up all required software to run the testsuite. GNU Parallel is pretty well tested on many systems. The testsuite is not - not even close, so expect it to puke errors all over the place. /Ole
