[Bug testsuite/103324] RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324 --- Comment #11 from Eric Gallager --- (In reply to Eric Gallager from comment #10) > (In reply to Richard Biener from comment #2) > > Thus, > > > > make check RUNTESTFLAGS="execute.exp" > > make check RUNTESTFLAGS="dg-torture.exp" > > > Just confirming that the proper way to combine these would be: > make check RUNTESTFLAGS="execute.exp dg-torture.exp" > ...correct? > (The docs said something about being whitespace-delimited, but I wasn't > quite sure if that just applied to the globbing part, or also to the .exp > filenames...) (ok, from testing, it appears that that's correct...)
[Bug testsuite/103324] RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324 --- Comment #10 from Eric Gallager --- (In reply to Richard Biener from comment #2) > Thus, > > make check RUNTESTFLAGS="execute.exp" > make check RUNTESTFLAGS="dg-torture.exp" Just confirming that the proper way to combine these would be: make check RUNTESTFLAGS="execute.exp dg-torture.exp" ...correct? (The docs said something about being whitespace-delimited, but I wasn't quite sure if that just applied to the globbing part, or also to the .exp filenames...)
[Bug testsuite/103324] RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324 --- Comment #9 from Eric Gallager --- (In reply to Sam James from comment #8) > Using make synchronisation can help a bit: > https://www.gnu.org/software/make/manual/html_node/Parallel-Output.html. > It's made our build logs in Gentoo a lot more readable for GCC, FWIW. So, I'm finally getting around to trying this, and it makes it seem as if the testsuite is hanging while waiting for output to be synchronized...
[Bug testsuite/103324] RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324 --- Comment #8 from Sam James --- Using make synchronisation can help a bit: https://www.gnu.org/software/make/manual/html_node/Parallel-Output.html. It's made our build logs in Gentoo a lot more readable for GCC, FWIW. As for the bug request: yes, this would be most helpful in trying to quickly ascertain if things work well enough as a non-GCC developer. Otherwise I'm stuck trying to diff failures and runs taking hours at a time. Not a complaint about your workflow, but saying there's a good usecase for some partial "really should pass" tests.
[Bug testsuite/103324] RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324 --- Comment #7 from Jonathan Wakely --- (In reply to Eric Gallager from comment #5) > So, now I'm running the testsuite anyways for other reasons, and one more > thing to note is that using any sort of parallelism when running the > testsuite (which is pretty much a must these days) makes picking out the > ‘Running … .exp’ lines more difficult than necessary... Surely only if you try to get them while the tests are still running? After they finish, all the output is flattened out into the .log files.
[Bug testsuite/103324] RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324 --- Comment #6 from Eric Gallager --- (In reply to Eric Gallager from comment #5) > (In reply to Eric Gallager from comment #3) > > https://gcc.gnu.org/install/test.html says "To get a list of the possible > > *.exp files, pipe the output of ‘make check’ into a file and look at the > > ‘Running … .exp’ lines." ...has anyone stored their output from doing so > > recently? I don't really want to run the entire testsuite just to generate > > this list... > > So, now I'm running the testsuite anyways for other reasons, and one more > thing to note is that using any sort of parallelism when running the > testsuite (which is pretty much a must these days) makes picking out the > ‘Running … .exp’ lines more difficult than necessary... In any case, here's my current list of "Running … .exp ..." lines (sorted and uniq-ed): Running ../../../../libatomic/testsuite/libatomic.c/c.exp ... Running ../../../../libgomp/testsuite/libgomp.c++/c++.exp ... Running ../../../../libgomp/testsuite/libgomp.c/c.exp ... Running ../../../../libgomp/testsuite/libgomp.fortran/fortran.exp ... Running ../../../../libgomp/testsuite/libgomp.graphite/graphite.exp ... Running ../../../../libgomp/testsuite/libgomp.oacc-c++/c++.exp ... Running ../../../../libgomp/testsuite/libgomp.oacc-c/c.exp ... Running ../../../../libgomp/testsuite/libgomp.oacc-fortran/fortran.exp ... Running ../../../../libitm/testsuite/libitm.c++/c++.exp ... Running ../../../../libitm/testsuite/libitm.c/c.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/analyzer/analyzer.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/asan/asan.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/bprob/bprob.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/charset/charset.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/compat/compat.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/compat/struct-layout-1.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/coroutines/coroutines.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/coroutines/torture/coro-torture.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/debug/debug.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/debug/dwarf2/dwarf2.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/dfp/dfp.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/dg.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/gcov/gcov.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/goacc-gomp/goacc-gomp.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/goacc/goacc.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/gomp/gomp.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/graphite/graphite.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/guality/guality.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/hwasan/hwasan.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/lto/lto.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/modules/modules.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/pch/pch.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/plugin/plugin.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/simulate-thread/simulate-thread.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/special/ecos.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/tls/tls.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/tm/tm.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/torture/dg-torture.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/torture/stackalign/stackalign.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/tree-prof/tree-prof.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/tsan/tsan.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/ubsan/ubsan.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.dg/vect/vect.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.old-deja/old-deja.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.target/aarch64/aarch64.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.target/aarch64/advsimd-intrinsics/advsimd-intrinsics.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.target/aarch64/sve/aarch64-sve.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.target/aarch64/sve/acle/aarch64-sve-acle-asm.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.target/aarch64/sve/acle/aarch64-sve-acle.exp ... Running /Users/ericgallager/gcc_newgit/gcc/testsuite/g++.target/aarch64/sve2/acle/aarch64-sve2-acle-asm.exp ... Running
[Bug testsuite/103324] RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324 --- Comment #5 from Eric Gallager --- (In reply to Eric Gallager from comment #3) > https://gcc.gnu.org/install/test.html says "To get a list of the possible > *.exp files, pipe the output of ‘make check’ into a file and look at the > ‘Running … .exp’ lines." ...has anyone stored their output from doing so > recently? I don't really want to run the entire testsuite just to generate > this list... So, now I'm running the testsuite anyways for other reasons, and one more thing to note is that using any sort of parallelism when running the testsuite (which is pretty much a must these days) makes picking out the ‘Running … .exp’ lines more difficult than necessary...
[Bug testsuite/103324] RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324 --- Comment #4 from Eric Gallager --- Another thing that would be useful would be to have (more) comments in the source code saying stuff like "/* this codepath is tested by */" or something... although I guess it could be a problem keeping them in sync...
[Bug testsuite/103324] RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324 --- Comment #3 from Eric Gallager --- https://gcc.gnu.org/install/test.html says "To get a list of the possible *.exp files, pipe the output of ‘make check’ into a file and look at the ‘Running … .exp’ lines." ...has anyone stored their output from doing so recently? I don't really want to run the entire testsuite just to generate this list...
[Bug testsuite/103324] RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324 --- Comment #2 from Richard Biener --- I'm usually running pieces that affect the area I am patching like vect.exp for vectorizer stuff. Generally a smoke test would be dg-torture.exp (runs C and C++ pieces) and execute.exp (C, ObjC, Go and Fortran). Thus, make check RUNTESTFLAGS="execute.exp" make check RUNTESTFLAGS="dg-torture.exp" for crosses compile.exp might be more light weight than execute.exp. In reality bootstrap itself should be smoke test enough ...
[Bug testsuite/103324] RFE: Add a `make quickcheck` or `make smoketest` Makefile target to allow only running a portion of the testsuite
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103324 Andrew Pinski changed: What|Removed |Added Last reconfirmed||2021-11-18 Ever confirmed|0 |1 Status|UNCONFIRMED |NEW --- Comment #1 from Andrew Pinski --- Confirmed, This what have been running since 2010 (and was there for a few years before that): cd check-check-dir; $(RUNTEST) TMPDIR=`pwd` --all --tools gcc $(DGFLAGS) \ --srcdir=$(SRC)/gcc/testsuite execute.exp=2112-1.c This was for an out of tree already built toolchain testing even. If this fails, I don't run the full testsuite. (this is not what I run for my upstream testing though, only for my internal toolchain testing).
