Re: Pike testing

[Bertrand Lupart]

Yeah, it maybe wasn’t engineered this way, but we end up with a shell script 
hidden in the source tree, depending on m4 you wouldn’t run at runtime. A 
system subtle enough to run tests against the Pike binary you just compiled 
instead of the installed one. A marvel for a Pike core hacker :)

Anyway, depending on m4 would disqualify any tool expected to be run on runtime 
or end-user side.

As you suggest, providing testsuite files in the Pike module source code would 
be the way to go. A pike -x something would make things easier, and a proper 
CI/CD pipeline could handle the burden.


Bertrand


> On 8 Sep 2023, at 07:54, H. William Welliver  wrote:
> 
> Well, mktestsuite is just a shell script that runs m4. There isn’t anything 
> that’s particularly “compile-time centric about it”. And certainly test_pike 
> could be modified to automatically update the test suite file if needed.
> 
> The m4 dependency means that you wouldn’t necessarily be guaranteed to be 
> able to generate testsuites on any machine that merely had pike installed 
> (Windows in particular comes to mind).
> 
> If you’re distributing code as an archive, you could always generate the 
> testsuite as part of the archive generation process. 
> 
> I use test suites to test some of my own code and while I use makefiles to 
> automate it, there’s no reason it needs to be that way. Making test_pike 
> regenerate the test suite if necessary would make it even more seamless. 
> 
> Even if test_pike weren’t modified to run mktestsuite when necessary, it 
> might be nice if test_pike noticed that a testsuite.in had been modified and 
> made a note of it in the output.
> 
> Bill

Re: Pike testing

[H. William Welliver]

Well, mktestsuite is just a shell script that runs m4. There isn’t anything 
that’s particularly “compile-time centric about it”. And certainly test_pike 
could be modified to automatically update the test suite file if needed.

The m4 dependency means that you wouldn’t necessarily be guaranteed to be able 
to generate testsuites on any machine that merely had pike installed (Windows 
in particular comes to mind).

If you’re distributing code as an archive, you could always generate the 
testsuite as part of the archive generation process. 

I use test suites to test some of my own code and while I use makefiles to 
automate it, there’s no reason it needs to be that way. Making test_pike 
regenerate the test suite if necessary would make it even more seamless. 

Even if test_pike weren’t modified to run mktestsuite when necessary, it might 
be nice if test_pike noticed that a testsuite.in had been modified and made a 
note of it in the output.

Bill

> On Sep 7, 2023, at 4:56 PM, Bertrand Lupart  wrote:
> 
> Hello,
> 
> 
>>> As a side note, maybe the brew Pike formula could be enhanced to
>>> provide mktestsuite in the PATH ?
>> 
>> I'm not sure what the best solution is for this; "mktestsuite" is kind of a 
>> generic name. Maybe we could add an option to test_pike to generate the 
>> testsuite?
> 
> Getting back some attention about this subject. The process of generating 
> testsuite files from testsuites.in ones would look like more consistent to me 
> if invoked from pike -x something
> 
> 
> However, I understand this testsuite stuff is compile-time-Pike-centric 
> (makefile and such), and I’m looking for something more run-time'ish, that 
> would test your own Pike module against a pre-installed Pike interpreter.
> 
> 1- testsuite.in file are distributed alongside own Pike module code
> 2- on the system likely to run the module, pike -x something runs the tests 
> (and probably need to generate/update testsuite files on the fly)
> 
> 
> Are testsuite suited for that, or should i look elsewhere ? Pest [1] comes to 
> mind
> 
> 
> Thank you,
> 
> 
> Bertrand
> 
> 
> [1] https://github.com/poppa/pest 
> 

Re: Pike testing

[Bertrand Lupart]

Hello,


>> As a side note, maybe the brew Pike formula could be enhanced to
>> provide mktestsuite in the PATH ?
> 
> I'm not sure what the best solution is for this; "mktestsuite" is kind of a 
> generic name. Maybe we could add an option to test_pike to generate the 
> testsuite?

Getting back some attention about this subject. The process of generating 
testsuite files from testsuites.in ones would look like more consistent to me 
if invoked from pike -x something


However, I understand this testsuite stuff is compile-time-Pike-centric 
(makefile and such), and I’m looking for something more run-time'ish, that 
would test your own Pike module against a pre-installed Pike interpreter.

1- testsuite.in file are distributed alongside own Pike module code
2- on the system likely to run the module, pike -x something runs the tests 
(and probably need to generate/update testsuite files on the fly)


Are testsuite suited for that, or should i look elsewhere ? Pest [1] comes to 
mind


Thank you,


Bertrand


[1] https://github.com/poppa/pest

Re: Pike testing

[william]

As a side note, maybe the brew Pike formula could be enhanced to
provide mktestsuite in the PATH ?



I'm not sure what the best solution is for this; "mktestsuite" is kind 
of a generic name. Maybe we could add an option to test_pike to generate 
the testsuite?



If some kind of interest, here’s an example of a CI that will run
the testsuite for a Pike module against latest Pike 7.8 and 8.0 Docker
images upon a push on the GitHub repo :
https://github.com/bertrand-lupart/pike-public-standards-csv/blob/master/.github/workflows/test-module.yml

Re: Pike testing

[Bertrand Lupart]

> pike  -x test_pike is a bundled tool for running testsuites (aka unit tests). 
> I’m pretty sure the test_pike.pike script described in the link is 
> essentially the same thing. I use it primarily in my standalone modules[1], 
> but it certainly isn’t limited to that. I think you also need to use 
> mktestsuite (primarily to expand the m4 macros (test_any and friends) to 
> generate testsuite files compatible with the test runner. I have some notes 
> about writing test suites[2] that expands on the stuff in the manual.
> 
> On a semi-related note, there are some messages in the archives about 
> assertions and some code that others (Per?) had put together to perform them.
> 
> Hope this helps!
> 
> Bill
> 
> [1] 
> https://hg.sr.ht/~hww3/pike_modules-public_parser_xml2/browse/testsuite.in?rev=tip
>  
> 
> [2] http://wiki.gotpike.org/PikeDevel%2FWriting%20Testsuites 
> 
> [3] 
> http://www.gotpike.org/pikemlarchives/search.pike?query=assert+boilerplate=25
>  
> 

Got the missing bits from the wiki, basically :
testsuite.in -> mktestsuite -> testsuite -> pike -x test_pike

Thank you !

As a side note, maybe the brew Pike formula could be enhanced to provide 
mktestsuite in the PATH ?

If some kind of interest, here’s an example of a CI that will run the testsuite 
for a Pike module against latest Pike 7.8 and 8.0 Docker images upon a push on 
the GitHub repo :
https://github.com/bertrand-lupart/pike-public-standards-csv/blob/master/.github/workflows/test-module.yml