Hi Ole!
Thanks for the reply.
On Tue, 29 Sep 2020 01:18:31 +0200
Ole Tange wrote:
> On Thu, Sep 24, 2020 at 8:37 AM Shlomi Fish wrote:
> >
> > Can the automated tests be included in the source tarball? It does not
> > include them currently which makes assuring that it works properly harder.
>
> They are available in git.
>
> I am not sure they would make sense to normal users, thus I am not
> convinced they should go into the tarball.
>
> Unfortunately some of them are heavily dependent on running on the
> same equipment as mine due to timing issues (and every time I change
> hardware or upgrade software, I have to tweak the tests or results a
> little).
>
I see. I suppose you are aware of running on a pristine CI system, e.g:
https://github.com/shlomif/Freenode-programming-channel-FAQ/blob/master/FAQ_with_ToC__generated.md#what-do-continuous-integration-ci-services-such-as-travis-ci-jenkins-or-appveyor-provide
> But it would be a good idea to identify those that are not dependent
> on this and make it clear that these should be able to run on all
> systems. And if possible change more tests to be independent.
Yes.
>
> If you want to help, a good start would be to have
> testsuite/REQUIREMENTS ported so it would work on more systems. It
> (tries to) install and set up all required software to run the
> testsuite.
Wow, it looks intimidating. And I thought some of my FOSS projects had too many
prereqs! Of course, some users will still complain even if there is a moderate
number of deps, which are even explicitly specified in the CI
configuration (e.g.: https://github.com/shlomif/fortune-mod/issues/44 ).
Anyway, one approach I used to avoid failing tests with certain
configurations/missing-deps is to specify "tags" / "labels" in an environment
variable and skip the test scripts (or individual assertions) if they are set
or unset, e.g in:
https://github.com/shlomif/fc-solve/blob/17426fe265a348fe1cd1df65d7fb1e1602711c86/fc-solve/source/t/t/fc-solve-output-Test-Trap.t#L207
(short url: https://is.gd/wyDrck )
>
> GNU Parallel is pretty well tested on many systems. The testsuite is
> not - not even close, so expect it to puke errors all over the place.
>
>
> /Ole
>
Thanks!
--
Shlomi Fish https://www.shlomifish.org/
Interview with Ben Collins-Sussman - https://shlom.in/sussman
I’m trying to achieve world peace and this regex is the last thing
standing in my way! ;)
Please reply to list if it's a mailing list post - https://shlom.in/reply .
