Branch: refs/heads/smoke-me/jkeenan/t-harness-revamp-20261008
  Home:   https://github.com/Perl/perl5
  Commit: 4e00d32ab156a13eb36f3c57c9936fb4e67c5a0e
      
https://github.com/Perl/perl5/commit/4e00d32ab156a13eb36f3c57c9936fb4e67c5a0e
  Author: James E Keenan <[email protected]>
  Date:   2026-10-12 (Mon, 12 Oct 2026)

  Changed paths:
    M t/TEST
    M t/harness
    M t/porting/test_testlist.t

  Log Message:
  -----------
  t/TEST: Change /dump_?tests/ to 'dump_manifested_tests'

Currently, the '-dumptests' switch in t/TEST only reports tests
listed in the MANIFEST.  In preparation for the addition/restoration of
tests generated via 'make' to the list of tests run by t/TEST, we need
to name things more precisely.

t/harness: For consistency with t/TEST, rename '@tests' and similar to
'@manifested_tests,' i.e., tests listed in MANIFEST.


  Commit: bfaf4b1733d0ee7abf603ec802c965e66cc71ba2
      
https://github.com/Perl/perl5/commit/bfaf4b1733d0ee7abf603ec802c965e66cc71ba2
  Author: James E Keenan <[email protected]>
  Date:   2026-10-12 (Mon, 12 Oct 2026)

  Changed paths:
    M t/TEST

  Log Message:
  -----------
  Rather than work on global @ARGV, clone and rename it

Currently, @ARGV starts out with a mixture of command-line switches and files to
be tested.  We then do lots of pushing onto it and re-writing elements.
This means that during debugging saying x \@ARGV doesn't show us what we
originally passed to the program.  Instead, assign to a lexically-scoped
variable.  Rename some other variables similarly.

Bring other aspects of t/TEST up to at least 2005-level best practices,
e.g., use lexical file and directory handles; use 3-argument 'open'.

Use 3-argument 'open'.


  Commit: 9b53b4fb1e3e301c82d2757dfac2b75ff8293057
      
https://github.com/Perl/perl5/commit/9b53b4fb1e3e301c82d2757dfac2b75ff8293057
  Author: James E Keenan <[email protected]>
  Date:   2026-10-12 (Mon, 12 Oct 2026)

  Changed paths:
    M t/TEST

  Log Message:
  -----------
  Add generated test files to list to be tested


  Commit: d1193a56681892ebadd0cffc40d29005d6c58caa
      
https://github.com/Perl/perl5/commit/d1193a56681892ebadd0cffc40d29005d6c58caa
  Author: James E Keenan <[email protected]>
  Date:   2026-10-12 (Mon, 12 Oct 2026)

  Changed paths:
    M t/TEST

  Log Message:
  -----------
  Corrections to t/TEST only

Leaving t/harness unchanged.

In t/harness, to change @manifested_tests (back) to @tests (making
similar changes to dump_manifested_tests) causes test failures in
t/porting/test_testlist.t.

So this commit makes changes in t/TEST suggested during code review but,
for the time being, leaves t/harness untouched.

On the one hand, that enables t/porting/test_testlist.t to PASS.  It
also keeps the generated tests from running when you are running
t/porting/test_testlist.t

On the other hand, it means that certain identifiers within t/harness
are misleadingly named.  As Tony pointed out in code review, the list of
tests accumulated during t/harness are not gleaned from the MANIFEST
and, hence, do not really qualify as "manifested" tests.

During development of GH #24918 I was trying to leave
t/porting/test_testlist.t as untouched as possible.  That was because
that test program, introduced in 2023, had as its objective:

    # Test that t/TEST and t/harness test the same files, and that all the
    # test files (.t files) listed in MANIFEST are tested by both.

To achieve that objective, test_testlist.t relies on the assumption that
each of t/TEST and t/harness have a command-line switch named
`-dumptests` that will produced the same results.


  Commit: 8ec9806425cc703077fe42d1b62517915817b770
      
https://github.com/Perl/perl5/commit/8ec9806425cc703077fe42d1b62517915817b770
  Author: James E Keenan <[email protected]>
  Date:   2026-10-12 (Mon, 12 Oct 2026)

  Changed paths:
    M t/harness
    M t/porting/test_testlist.t

  Log Message:
  -----------
  test_testlist.t: Temporarily skip all tests

So as to accommodate reversion of 'manifested_tests' in t/harness to
just 'tests' ... and that's so as not to get in our way when
implementing generated tests in t/harness.


  Commit: e908c8f301d9fdd4e7c874057ebebe5d07ab58e6
      
https://github.com/Perl/perl5/commit/e908c8f301d9fdd4e7c874057ebebe5d07ab58e6
  Author: James E Keenan <[email protected]>
  Date:   2026-10-12 (Mon, 12 Oct 2026)

  Changed paths:
    M t/TEST

  Log Message:
  -----------
  Restore name of dump_manifested_tests() to dump_tests()

There's nothing within this subroutine that is specific to
manifested_tests.  If we simply call it dump_tests() we can use it
within t/harness without redefining it.


  Commit: f658de62d3823d04ea69a11496c4c19386ee854d
      
https://github.com/Perl/perl5/commit/f658de62d3823d04ea69a11496c4c19386ee854d
  Author: James E Keenan <[email protected]>
  Date:   2026-10-12 (Mon, 12 Oct 2026)

  Changed paths:
    M t/harness

  Log Message:
  -----------
  Uncuddle some else's

Add inline comments where we're pulling in subroutines defined in
t/TEST.

Some other small tidying.


  Commit: 526785370aa4ef497bff165cb24e71c0bf80e572
      
https://github.com/Perl/perl5/commit/526785370aa4ef497bff165cb24e71c0bf80e572
  Author: James E Keenan <[email protected]>
  Date:   2026-10-12 (Mon, 12 Oct 2026)

  Changed paths:
    M t/harness

  Log Message:
  -----------
  Move declaration of variable closer to where used


  Commit: b8e3389b88066dfce587b4c6925839058daa08b2
      
https://github.com/Perl/perl5/commit/b8e3389b88066dfce587b4c6925839058daa08b2
  Author: James E Keenan <[email protected]>
  Date:   2026-10-12 (Mon, 12 Oct 2026)

  Changed paths:
    M t/harness
    M t/porting/test_testlist.t

  Log Message:
  -----------
  Reactivate (simpler) t/porting/test_testlist.t

At this point, we're comparing the manifested tests found by t/TEST to
match all the tests found by t/harness.  We have not yet implemented
testing of generated tests in t/harness, so we're not yet concerned with
the number or identity of generated tests found by t/TEST.


  Commit: f86059118edcda0ca112d2a712906b94e870fa11
      
https://github.com/Perl/perl5/commit/f86059118edcda0ca112d2a712906b94e870fa11
  Author: James E Keenan <[email protected]>
  Date:   2026-10-12 (Mon, 12 Oct 2026)

  Changed paths:
    M t/harness

  Log Message:
  -----------
  t/harness: Locate generated tests and test them


  Commit: 22ddd68daac80853b30174582191384f977559d5
      
https://github.com/Perl/perl5/commit/22ddd68daac80853b30174582191384f977559d5
  Author: James E Keenan <[email protected]>
  Date:   2026-10-12 (Mon, 12 Oct 2026)

  Changed paths:
    M t/harness

  Log Message:
  -----------
  Don't mix specific tests with generated tests


  Commit: 7944b385213fab293d84cfbb3a5539d4651a5d0a
      
https://github.com/Perl/perl5/commit/7944b385213fab293d84cfbb3a5539d4651a5d0a
  Author: James E Keenan <[email protected]>
  Date:   2026-10-12 (Mon, 12 Oct 2026)

  Changed paths:
    M t/porting/test_testlist.t

  Log Message:
  -----------
  Pause test_testlist.t temporarily


  Commit: 5255e7f37cb86e8ed9e30f7beaa35040b3202070
      
https://github.com/Perl/perl5/commit/5255e7f37cb86e8ed9e30f7beaa35040b3202070
  Author: James E Keenan <[email protected]>
  Date:   2026-10-12 (Mon, 12 Oct 2026)

  Changed paths:
    M t/TEST

  Log Message:
  -----------
  Reposition identification of generated tests


  Commit: 9c40525b97a2a29a195eff6b42657c176589aa13
      
https://github.com/Perl/perl5/commit/9c40525b97a2a29a195eff6b42657c176589aa13
  Author: James E Keenan <[email protected]>
  Date:   2026-10-12 (Mon, 12 Oct 2026)

  Changed paths:
    M t/harness

  Log Message:
  -----------
  _compute_tests_and_ordering:  Improve encapsulation

%total_time was used only within _compute_tests_and_ordering() and
sort_by_execution_order.  The latter, in turn, was/is only called within
_compute_tests_and_ordering().  Bring declaration of %total_time inside
_compute_tests_and_ordering().  Transform  sort_by_execution_order()
into a lexical sub declared within _compute_tests_and_ordering().


  Commit: 7c2d44ac80e01354898710e46b1355649b6371b2
      
https://github.com/Perl/perl5/commit/7c2d44ac80e01354898710e46b1355649b6371b2
  Author: James E Keenan <[email protected]>
  Date:   2026-10-12 (Mon, 12 Oct 2026)

  Changed paths:
    M t/porting/test_testlist.t

  Log Message:
  -----------
  test_testlib.t: Add testing subroutines and use them

Add 3 subroutines which can be used to test a given list of test files
via t/TEST, t/harness and ./runtests choose, respectively.

Add 2 subroutines used to create lookup tables within the 3
aforementioned subroutines.

Rearrange the file so that subroutines are defined beneath the test code
and superseded tests are placed in an END block.  (We'll get back to
them later.)


  Commit: 3eb1c8dc8c242d0058814cf3f063ef08368a085c
      
https://github.com/Perl/perl5/commit/3eb1c8dc8c242d0058814cf3f063ef08368a085c
  Author: James E Keenan <[email protected]>
  Date:   2026-10-12 (Mon, 12 Oct 2026)

  Changed paths:
    M t/porting/test_testlist.t

  Log Message:
  -----------
  Test generated tests for dist/Devel-PPPort

Using recently developed testing subroutines.  Add inline comments;
deleted commented-out code.


  Commit: 2d1b4fb07b4103960d96db768126403a70a0c084
      
https://github.com/Perl/perl5/commit/2d1b4fb07b4103960d96db768126403a70a0c084
  Author: James E Keenan <[email protected]>
  Date:   2026-10-12 (Mon, 12 Oct 2026)

  Changed paths:
    M t/porting/test_testlist.t

  Log Message:
  -----------
  test_testlist.t: Restore pre-5.43.4 tests

Suitably tweaked.


  Commit: d0c8746e1fadb249d13e9d3494e1583378d73056
      
https://github.com/Perl/perl5/commit/d0c8746e1fadb249d13e9d3494e1583378d73056
  Author: James E Keenan <[email protected]>
  Date:   2026-10-12 (Mon, 12 Oct 2026)

  Changed paths:
    M t/porting/test_testlist.t

  Log Message:
  -----------
  Skip two instances of 'test_via_runtests'

Experimentation showed that while the test suite ran as expected when invoked
with 'make test', 'make test_harness', etc., we experienced indefinite hangs
during 't/porting/test_testlist.t' whenever we called the following:

    $ make test_porting

    # what's "inside" 'make test_porting':
    $ TEST_ARGS='porting/*.t lib/diagnostics.t' TESTFILE=harness ./runtests 
choose

    $ TEST_ARGS='porting/test_testlist.t' TESTFILE=TEST ./runtests choose

    $ TEST_ARGS='porting/*.t' TESTFILE=TEST ./runtests choose

Examination of the output of 'ps auxwww | grep testlist' showed that
additional processes were being spawned.

But the following did *not* hang:

    $ TESTFILE=harness ./runtests choose

    $ TESTFILE=TEST ./runtests choose

It is not clear why passing "TEST_ARGS=[something which expands to include
porting/test_testlist.t]" to "./runtests choose" behaves in this way.  But as
that is not a way most users would trigger these tests, we can defer further
investigation for now via 'todo_skip'.


  Commit: 1d9522a23e7efc32b757aa58ee46169bed897101
      
https://github.com/Perl/perl5/commit/1d9522a23e7efc32b757aa58ee46169bed897101
  Author: James E Keenan <[email protected]>
  Date:   2026-10-12 (Mon, 12 Oct 2026)

  Changed paths:
    M t/porting/test_testlist.t

  Log Message:
  -----------
  Avoid uninitialized value warning


  Commit: b3d5738bc50734b9e5a46904dcff75973a5c0577
      
https://github.com/Perl/perl5/commit/b3d5738bc50734b9e5a46904dcff75973a5c0577
  Author: James E Keenan <[email protected]>
  Date:   2026-10-12 (Mon, 12 Oct 2026)

  Changed paths:
    M t/porting/test_testlist.t

  Log Message:
  -----------
  test_testlist.t:  Skip 4 tests on Win32

We need to determine cause of test failures on that platform.


Compare: https://github.com/Perl/perl5/compare/b32d53bd0716...b3d5738bc507

To unsubscribe from these emails, change your notification settings at 
https://github.com/Perl/perl5/settings/notifications

Reply via email to