Branch: refs/heads/smoke-me/jkeenan/t-harness-revamp-20261008
Home: https://github.com/Perl/perl5
Commit: 4e00d32ab156a13eb36f3c57c9936fb4e67c5a0e
https://github.com/Perl/perl5/commit/4e00d32ab156a13eb36f3c57c9936fb4e67c5a0e
Author: James E Keenan <[email protected]>
Date: 2026-10-12 (Mon, 12 Oct 2026)
Changed paths:
M t/TEST
M t/harness
M t/porting/test_testlist.t
Log Message:
-----------
t/TEST: Change /dump_?tests/ to 'dump_manifested_tests'
Currently, the '-dumptests' switch in t/TEST only reports tests
listed in the MANIFEST. In preparation for the addition/restoration of
tests generated via 'make' to the list of tests run by t/TEST, we need
to name things more precisely.
t/harness: For consistency with t/TEST, rename '@tests' and similar to
'@manifested_tests,' i.e., tests listed in MANIFEST.
Commit: bfaf4b1733d0ee7abf603ec802c965e66cc71ba2
https://github.com/Perl/perl5/commit/bfaf4b1733d0ee7abf603ec802c965e66cc71ba2
Author: James E Keenan <[email protected]>
Date: 2026-10-12 (Mon, 12 Oct 2026)
Changed paths:
M t/TEST
Log Message:
-----------
Rather than work on global @ARGV, clone and rename it
Currently, @ARGV starts out with a mixture of command-line switches and files to
be tested. We then do lots of pushing onto it and re-writing elements.
This means that during debugging saying x \@ARGV doesn't show us what we
originally passed to the program. Instead, assign to a lexically-scoped
variable. Rename some other variables similarly.
Bring other aspects of t/TEST up to at least 2005-level best practices,
e.g., use lexical file and directory handles; use 3-argument 'open'.
Use 3-argument 'open'.
Commit: 9b53b4fb1e3e301c82d2757dfac2b75ff8293057
https://github.com/Perl/perl5/commit/9b53b4fb1e3e301c82d2757dfac2b75ff8293057
Author: James E Keenan <[email protected]>
Date: 2026-10-12 (Mon, 12 Oct 2026)
Changed paths:
M t/TEST
Log Message:
-----------
Add generated test files to list to be tested
Commit: d1193a56681892ebadd0cffc40d29005d6c58caa
https://github.com/Perl/perl5/commit/d1193a56681892ebadd0cffc40d29005d6c58caa
Author: James E Keenan <[email protected]>
Date: 2026-10-12 (Mon, 12 Oct 2026)
Changed paths:
M t/TEST
Log Message:
-----------
Corrections to t/TEST only
Leaving t/harness unchanged.
In t/harness, to change @manifested_tests (back) to @tests (making
similar changes to dump_manifested_tests) causes test failures in
t/porting/test_testlist.t.
So this commit makes changes in t/TEST suggested during code review but,
for the time being, leaves t/harness untouched.
On the one hand, that enables t/porting/test_testlist.t to PASS. It
also keeps the generated tests from running when you are running
t/porting/test_testlist.t
On the other hand, it means that certain identifiers within t/harness
are misleadingly named. As Tony pointed out in code review, the list of
tests accumulated during t/harness are not gleaned from the MANIFEST
and, hence, do not really qualify as "manifested" tests.
During development of GH #24918 I was trying to leave
t/porting/test_testlist.t as untouched as possible. That was because
that test program, introduced in 2023, had as its objective:
# Test that t/TEST and t/harness test the same files, and that all the
# test files (.t files) listed in MANIFEST are tested by both.
To achieve that objective, test_testlist.t relies on the assumption that
each of t/TEST and t/harness have a command-line switch named
`-dumptests` that will produced the same results.
Commit: 8ec9806425cc703077fe42d1b62517915817b770
https://github.com/Perl/perl5/commit/8ec9806425cc703077fe42d1b62517915817b770
Author: James E Keenan <[email protected]>
Date: 2026-10-12 (Mon, 12 Oct 2026)
Changed paths:
M t/harness
M t/porting/test_testlist.t
Log Message:
-----------
test_testlist.t: Temporarily skip all tests
So as to accommodate reversion of 'manifested_tests' in t/harness to
just 'tests' ... and that's so as not to get in our way when
implementing generated tests in t/harness.
Commit: e908c8f301d9fdd4e7c874057ebebe5d07ab58e6
https://github.com/Perl/perl5/commit/e908c8f301d9fdd4e7c874057ebebe5d07ab58e6
Author: James E Keenan <[email protected]>
Date: 2026-10-12 (Mon, 12 Oct 2026)
Changed paths:
M t/TEST
Log Message:
-----------
Restore name of dump_manifested_tests() to dump_tests()
There's nothing within this subroutine that is specific to
manifested_tests. If we simply call it dump_tests() we can use it
within t/harness without redefining it.
Commit: f658de62d3823d04ea69a11496c4c19386ee854d
https://github.com/Perl/perl5/commit/f658de62d3823d04ea69a11496c4c19386ee854d
Author: James E Keenan <[email protected]>
Date: 2026-10-12 (Mon, 12 Oct 2026)
Changed paths:
M t/harness
Log Message:
-----------
Uncuddle some else's
Add inline comments where we're pulling in subroutines defined in
t/TEST.
Some other small tidying.
Commit: 526785370aa4ef497bff165cb24e71c0bf80e572
https://github.com/Perl/perl5/commit/526785370aa4ef497bff165cb24e71c0bf80e572
Author: James E Keenan <[email protected]>
Date: 2026-10-12 (Mon, 12 Oct 2026)
Changed paths:
M t/harness
Log Message:
-----------
Move declaration of variable closer to where used
Commit: b8e3389b88066dfce587b4c6925839058daa08b2
https://github.com/Perl/perl5/commit/b8e3389b88066dfce587b4c6925839058daa08b2
Author: James E Keenan <[email protected]>
Date: 2026-10-12 (Mon, 12 Oct 2026)
Changed paths:
M t/harness
M t/porting/test_testlist.t
Log Message:
-----------
Reactivate (simpler) t/porting/test_testlist.t
At this point, we're comparing the manifested tests found by t/TEST to
match all the tests found by t/harness. We have not yet implemented
testing of generated tests in t/harness, so we're not yet concerned with
the number or identity of generated tests found by t/TEST.
Commit: f86059118edcda0ca112d2a712906b94e870fa11
https://github.com/Perl/perl5/commit/f86059118edcda0ca112d2a712906b94e870fa11
Author: James E Keenan <[email protected]>
Date: 2026-10-12 (Mon, 12 Oct 2026)
Changed paths:
M t/harness
Log Message:
-----------
t/harness: Locate generated tests and test them
Commit: 22ddd68daac80853b30174582191384f977559d5
https://github.com/Perl/perl5/commit/22ddd68daac80853b30174582191384f977559d5
Author: James E Keenan <[email protected]>
Date: 2026-10-12 (Mon, 12 Oct 2026)
Changed paths:
M t/harness
Log Message:
-----------
Don't mix specific tests with generated tests
Commit: 7944b385213fab293d84cfbb3a5539d4651a5d0a
https://github.com/Perl/perl5/commit/7944b385213fab293d84cfbb3a5539d4651a5d0a
Author: James E Keenan <[email protected]>
Date: 2026-10-12 (Mon, 12 Oct 2026)
Changed paths:
M t/porting/test_testlist.t
Log Message:
-----------
Pause test_testlist.t temporarily
Commit: 5255e7f37cb86e8ed9e30f7beaa35040b3202070
https://github.com/Perl/perl5/commit/5255e7f37cb86e8ed9e30f7beaa35040b3202070
Author: James E Keenan <[email protected]>
Date: 2026-10-12 (Mon, 12 Oct 2026)
Changed paths:
M t/TEST
Log Message:
-----------
Reposition identification of generated tests
Commit: 9c40525b97a2a29a195eff6b42657c176589aa13
https://github.com/Perl/perl5/commit/9c40525b97a2a29a195eff6b42657c176589aa13
Author: James E Keenan <[email protected]>
Date: 2026-10-12 (Mon, 12 Oct 2026)
Changed paths:
M t/harness
Log Message:
-----------
_compute_tests_and_ordering: Improve encapsulation
%total_time was used only within _compute_tests_and_ordering() and
sort_by_execution_order. The latter, in turn, was/is only called within
_compute_tests_and_ordering(). Bring declaration of %total_time inside
_compute_tests_and_ordering(). Transform sort_by_execution_order()
into a lexical sub declared within _compute_tests_and_ordering().
Commit: 7c2d44ac80e01354898710e46b1355649b6371b2
https://github.com/Perl/perl5/commit/7c2d44ac80e01354898710e46b1355649b6371b2
Author: James E Keenan <[email protected]>
Date: 2026-10-12 (Mon, 12 Oct 2026)
Changed paths:
M t/porting/test_testlist.t
Log Message:
-----------
test_testlib.t: Add testing subroutines and use them
Add 3 subroutines which can be used to test a given list of test files
via t/TEST, t/harness and ./runtests choose, respectively.
Add 2 subroutines used to create lookup tables within the 3
aforementioned subroutines.
Rearrange the file so that subroutines are defined beneath the test code
and superseded tests are placed in an END block. (We'll get back to
them later.)
Commit: 3eb1c8dc8c242d0058814cf3f063ef08368a085c
https://github.com/Perl/perl5/commit/3eb1c8dc8c242d0058814cf3f063ef08368a085c
Author: James E Keenan <[email protected]>
Date: 2026-10-12 (Mon, 12 Oct 2026)
Changed paths:
M t/porting/test_testlist.t
Log Message:
-----------
Test generated tests for dist/Devel-PPPort
Using recently developed testing subroutines. Add inline comments;
deleted commented-out code.
Commit: 2d1b4fb07b4103960d96db768126403a70a0c084
https://github.com/Perl/perl5/commit/2d1b4fb07b4103960d96db768126403a70a0c084
Author: James E Keenan <[email protected]>
Date: 2026-10-12 (Mon, 12 Oct 2026)
Changed paths:
M t/porting/test_testlist.t
Log Message:
-----------
test_testlist.t: Restore pre-5.43.4 tests
Suitably tweaked.
Commit: d0c8746e1fadb249d13e9d3494e1583378d73056
https://github.com/Perl/perl5/commit/d0c8746e1fadb249d13e9d3494e1583378d73056
Author: James E Keenan <[email protected]>
Date: 2026-10-12 (Mon, 12 Oct 2026)
Changed paths:
M t/porting/test_testlist.t
Log Message:
-----------
Skip two instances of 'test_via_runtests'
Experimentation showed that while the test suite ran as expected when invoked
with 'make test', 'make test_harness', etc., we experienced indefinite hangs
during 't/porting/test_testlist.t' whenever we called the following:
$ make test_porting
# what's "inside" 'make test_porting':
$ TEST_ARGS='porting/*.t lib/diagnostics.t' TESTFILE=harness ./runtests
choose
$ TEST_ARGS='porting/test_testlist.t' TESTFILE=TEST ./runtests choose
$ TEST_ARGS='porting/*.t' TESTFILE=TEST ./runtests choose
Examination of the output of 'ps auxwww | grep testlist' showed that
additional processes were being spawned.
But the following did *not* hang:
$ TESTFILE=harness ./runtests choose
$ TESTFILE=TEST ./runtests choose
It is not clear why passing "TEST_ARGS=[something which expands to include
porting/test_testlist.t]" to "./runtests choose" behaves in this way. But as
that is not a way most users would trigger these tests, we can defer further
investigation for now via 'todo_skip'.
Commit: 1d9522a23e7efc32b757aa58ee46169bed897101
https://github.com/Perl/perl5/commit/1d9522a23e7efc32b757aa58ee46169bed897101
Author: James E Keenan <[email protected]>
Date: 2026-10-12 (Mon, 12 Oct 2026)
Changed paths:
M t/porting/test_testlist.t
Log Message:
-----------
Avoid uninitialized value warning
Commit: b3d5738bc50734b9e5a46904dcff75973a5c0577
https://github.com/Perl/perl5/commit/b3d5738bc50734b9e5a46904dcff75973a5c0577
Author: James E Keenan <[email protected]>
Date: 2026-10-12 (Mon, 12 Oct 2026)
Changed paths:
M t/porting/test_testlist.t
Log Message:
-----------
test_testlist.t: Skip 4 tests on Win32
We need to determine cause of test failures on that platform.
Compare: https://github.com/Perl/perl5/compare/b32d53bd0716...b3d5738bc507
To unsubscribe from these emails, change your notification settings at
https://github.com/Perl/perl5/settings/notifications