On Mon, Apr 07, 2025 at 09:30:59AM +0200, Richard Biener wrote:
> On Mon, Apr 7, 2025 at 9:00 AM Simon Sobisch <simonsobi...@gnu.org> wrote:
> > My question stands on integrating COBOLworx' UAT as-is for now
> > (Copyright is all on FSF; built automatically [it is autoconf, which is
> > a requirement for VCS checkouts], possibly also hooked into the current
> > test target) - with the goal to get rid of UAT later (next GCC version,
> > not GCC 15).
> >
> > There's also the question about integrating NIST into GCC upstream -
> > that is a subfolder and would only be executed upon explicit call by
> > maintainers (newcob.val / newcob.val.gz may be either included in VCS or
> > even downloaded manually...).
>
> As I repeatedly said I'd welcome a test harness like Ada ACATS for running
> the NIST testsuite plus a contrib/download_cobol_nist script that downloads
> the NIST file and prepares it for use. I'd suggest to, similar as with ACATS,
> have a separate make target for testing (but still invoked with make check,
> when present).
But it would be much better if the harness for NIST testing was in dejagnu
rather than anything else, only that can handle easily cross-compilations
with target boards, parallelization respecting make job reserve, seamless
result integration.
My understanding has been NIST is a single file from which some tool needs
to dig up individual testcases (tcl string support should be able to deal
with that), figuring out what options etc. to pass search test and from
somewhere find out the expected output for each test.
Jakub
