On Fri, 29 Aug 2025 at 11:35, Richard Earnshaw via Gcc <gcc@gcc.gnu.org> wrote: > > On 29/08/2025 04:08, Jacob Bachmeyer wrote: > > On 8/28/25 10:10, Jeff Law wrote: > >> On 8/28/25 8:09 AM, Richard Earnshaw (lists) wrote: > >>> On 28/08/2025 15:01, Iain Sandoe wrote: > >>>> [...] > >>> > >>> Well really, the compare-tests script should report duplicate results > >>> as a problem as well, since > >>> > >>> PASS: abcd > >>> ... > >>> PASS: abcd > >>> > >>> is just a dup pass/fail waiting to happen. > >> Yup. A duplicate testname should be reported. These cause major > >> headaches if one passes, but the other fails -- it looks like a > >> regression to the comparison scripting we have. > > > > The problem with detecting duplicate names in the DejaGnu framework is > > that it would add memory overhead that scales with the number of tests > > and DejaGnu tries to avoid that kind of unbounded space requirement. > > (OK, it *is* bounded for any finite testsuite, but the idea of a > > steadily growing memory footprint during a test run still bothers me.) > > > > I suggest that the comparison script GCC uses is probably the best place > > to check for duplicate test names, since that seems to also be the > > script that can be confused by them. > > > > That's exactly what I was suggesting. Trying to do it in dejagnu would > be a nightmare given that we run multiple instances of it to get > parallel testing. >
Which script do people use these days? Here is a quick patch for compare_tests, which actually detected duplicates ;-) I can commit that, if it helps. Thanks, Christophe > R. > > > > > -- Jacob > > > > >
diff --git a/contrib/compare_tests b/contrib/compare_tests index e09fc4f113a..98ffe3fcda3 100755 --- a/contrib/compare_tests +++ b/contrib/compare_tests @@ -41,6 +41,8 @@ tmp1=$TMPDIR/$tool-testing.$$a tmp2=$TMPDIR/$tool-testing.$$b now_s=$TMPDIR/$tool-testing.$$d before_s=$TMPDIR/$tool-testing.$$e +now_u=$TMPDIR/$tool-uniq.$$d +before_u=$TMPDIR/$tool-uniq.$$e lst1=$TMPDIR/$tool-lst1.$$ lst2=$TMPDIR/$tool-lst2.$$ lst3=$TMPDIR/$tool-lst3.$$ @@ -48,7 +50,7 @@ lst4=$TMPDIR/$tool-lst4.$$ lst5=$TMPDIR/$tool-lst5.$$ sum1=$TMPDIR/$tool-sum1.$$ sum2=$TMPDIR/$tool-sum2.$$ -tmps="$tmp1 $tmp2 $now_s $before_s $lst1 $lst2 $lst3 $lst4 $lst5 $sum1 $sum2" +tmps="$tmp1 $tmp2 $now_s $before_s $now_u $before_u $lst1 $lst2 $lst3 $lst4 $lst5 $sum1 $sum2" [ "$1" = "-strict" ] && strict=$1 && shift [ "$1" = "-?" ] && usage @@ -124,6 +126,22 @@ fi sort -t ':' $skip1 "$now" > "$now_s" sort -t ':' $skip1 "$before" > "$before_s" +uniq -c "$before_s" | grep -v '^ 1 ' > "$before_u" +if [ -f "$before_u" ]; then + echo "Non-unique test names before:" + cat "$before_u" + echo + exit_status=1 +fi + +uniq -c "$now_s" | grep -v '^ 1 ' > "$now_u" +if [ -f "$now_u" ]; then + echo "Non-unique test names now:" + cat "$now_u" + echo + exit_status=1 +fi + grep '^FAIL:' "$now_s" | sed 's/^[^:]*:[ ]//' >$tmp1 grep '^PASS' "$before_s" | sed 's/^[^:]*:[ ]//' | comm -12 $tmp1 - >$tmp2
