As a note, you can force test to run in multiple/specific standard by using:
> GLIBCXX_TESTSUITE_STDS=20,23 make check
On Thu, Jul 17, 2025 at 12:57 PM Jonathan Wakely <jwak...@redhat.com> wrote:
> On Thu, 17 Jul 2025 at 11:54, Jonathan Wakely <jwak...@redhat.com> wrote:
> >
> > On Thu, 17 Jul 2025 at 11:32, Luc Grosheintz wrote:
> > >
> > > Thank you! This uncovered a misconception of mine, namely that
> > >
> > > { target c++17 }
> > >
> > > would automatically run on C++17 and all later versions. Which is
> > > true, by accident, for { target c++23 }. At least in my rather
> > > default config it automatically also checks C++26 if the target
> > > larger than or equal to C++20.
> >
> > See proc v3-dg-runtest in testsuite/lib/libstdc++.exp which examines
> > the minimum C++ version required by the test (in this case C++17) and
> > decides whether that means it should be run with the implicit default
> > -std=gnu++17 or if it needs a newer standard to be explicitly enabled.
> > In the latter case, it *also* tests with the newest standard:
> >
> > # If the test requires a newer C++ version than which
> > # is tested by default, use that C++ version for that
> > # single test.
> >
> > So a test with { target c++17 } is happy with the default and is only
> > tested once.
> >
> > A test with { target c++20 } requires -std=gnu++20 (otherwise it will
> > be skipped as UNSUPPORTED) and to exercise newer code more thoroughly,
> > it gets tested a second time with -std=gnu++26
>
> Which is done by this bit of the proc:
>
> if { $min_std > $default_std } {
> set std_list $min_std
> if { $min_std != $max_std } {
> # Also test the latest version.
> lappend std_list "$max_std"
> }
> } else {
> # Only run each test once with the default -std option.
> # This avoids increasing the run time for most testers.
> # Libstdc++ developers can override this with v3_std_list.
> set std_list $default_std
> }
>
> So if the test says it needs X, where (X > c++17 && X != c++26), then add
> c++26.
>
>
> > > The reason I liked { target c++17 } is that it opens up the
> > > possibility to check that the macro for a C++23 feature is not
> > > set while compiling C++{17,20} code. Similarly, the same FTM can
> > > have different values for different versions of the standard; and
> > > I've seen examples that check these values. Therefore, I have the
> > > impression that testing with multiple versions of the standard is
> > > needed to check FTMs thoroughly.
> >
> > Strictly speaking, yes. But in practice we very rarely get the macro
> > values wrong since we switched to defining the macros in version.def
> > (instead of manually adding #define in individual headers, with
> > complex #if #else logic around them to get the values right). And if
> > we get them wrong, we're probably testing for the wrong values too :-)
> >
> > So I don't object to more thorough testing, but I don't feel it's
> essential.
> >
> > > I'm tempted to say that the code for FTMs is unlikely to break
> > > accidentally.
> >
> > That's true since GCC 14. In the bad old days we had problems like
> > https://gcc.gnu.org/PR105269 and the ones fixed by
> > https://gcc.gnu.org/g:9d63ce7c4c0a993c419dceb823e71cba5da99e24
> >
> > > Hence, thorough testing is needed when first
> > > implementing the FTM and then periodically. (Though invariably
> > > such a statement means the unlikely happens a few days later.)
> >
> > Ha, yes :-)
>
>
