Branch: refs/heads/smoke-me/jkeenan/t-harness-revamp-20261008
  Home:   https://github.com/Perl/perl5
  Commit: 1e24ecf3d90e82e2f526eccc5bb3abbcb93c619e
      
https://github.com/Perl/perl5/commit/1e24ecf3d90e82e2f526eccc5bb3abbcb93c619e
  Author: James E Keenan <[email protected]>
  Date:   2026-10-11 (Sun, 11 Oct 2026)

  Changed paths:
    M t/TEST
    M t/harness
    M t/porting/test_testlist.t

  Log Message:
  -----------
  t/TEST: Change /dump_?tests/ to 'dump_manifested_tests'

Currently, the '-dumptests' switch in t/TEST only reports tests
listed in the MANIFEST.  In preparation for the addition/restoration of
tests generated via 'make' to the list of tests run by t/TEST, we need
to name things more precisely.

t/harness: For consistency with t/TEST, rename '@tests' and similar to
'@manifested_tests,' i.e., tests listed in MANIFEST.


  Commit: 034a7bedc9e9667e2bb46a4409e1afcece9374e9
      
https://github.com/Perl/perl5/commit/034a7bedc9e9667e2bb46a4409e1afcece9374e9
  Author: James E Keenan <[email protected]>
  Date:   2026-10-11 (Sun, 11 Oct 2026)

  Changed paths:
    M t/TEST

  Log Message:
  -----------
  Rather than work on global @ARGV, clone and rename it

Currently, @ARGV starts out with a mixture of command-line switches and files to
be tested.  We then do lots of pushing onto it and re-writing elements.
This means that during debugging saying x \@ARGV doesn't show us what we
originally passed to the program.  Instead, assign to a lexically-scoped
variable.  Rename some other variables similarly.

Bring other aspects of t/TEST up to at least 2005-level best practices,
e.g., use lexical file and directory handles; use 3-argument 'open'.

Use 3-argument 'open'.


  Commit: 086529718d8cbc47687e7f0c6be5c55d04777ceb
      
https://github.com/Perl/perl5/commit/086529718d8cbc47687e7f0c6be5c55d04777ceb
  Author: James E Keenan <[email protected]>
  Date:   2026-10-11 (Sun, 11 Oct 2026)

  Changed paths:
    M t/TEST

  Log Message:
  -----------
  Add generated test files to list to be tested


  Commit: 25e83b441f2599138a89517ffc5e6fcc603928da
      
https://github.com/Perl/perl5/commit/25e83b441f2599138a89517ffc5e6fcc603928da
  Author: James E Keenan <[email protected]>
  Date:   2026-10-11 (Sun, 11 Oct 2026)

  Changed paths:
    M t/TEST

  Log Message:
  -----------
  Corrections to t/TEST only

Leaving t/harness unchanged.

In t/harness, to change @manifested_tests (back) to @tests (making
similar changes to dump_manifested_tests) causes test failures in
t/porting/test_testlist.t.

So this commit makes changes in t/TEST suggested during code review but,
for the time being, leaves t/harness untouched.

On the one hand, that enables t/porting/test_testlist.t to PASS.  It
also keeps the generated tests from running when you are running
t/porting/test_testlist.t

On the other hand, it means that certain identifiers within t/harness
are misleadingly named.  As Tony pointed out in code review, the list of
tests accumulated during t/harness are not gleaned from the MANIFEST
and, hence, do not really qualify as "manifested" tests.

During development of GH #24918 I was trying to leave
t/porting/test_testlist.t as untouched as possible.  That was because
that test program, introduced in 2023, had as its objective:

    # Test that t/TEST and t/harness test the same files, and that all the
    # test files (.t files) listed in MANIFEST are tested by both.

To achieve that objective, test_testlist.t relies on the assumption that
each of t/TEST and t/harness have a command-line switch named
`-dumptests` that will produced the same results.


  Commit: 1bfe8dd35ee4ae4fd878eea67542dc00b0fa6788
      
https://github.com/Perl/perl5/commit/1bfe8dd35ee4ae4fd878eea67542dc00b0fa6788
  Author: James E Keenan <[email protected]>
  Date:   2026-10-11 (Sun, 11 Oct 2026)

  Changed paths:
    M t/harness
    M t/porting/test_testlist.t

  Log Message:
  -----------
  test_testlist.t: Temporarily skip all tests

So as to accommodate reversion of 'manifested_tests' in t/harness to
just 'tests' ... and that's so as not to get in our way when
implementing generated tests in t/harness.


  Commit: d63eb4f1c2a53f1a9466d1933169114cb6dc64e4
      
https://github.com/Perl/perl5/commit/d63eb4f1c2a53f1a9466d1933169114cb6dc64e4
  Author: James E Keenan <[email protected]>
  Date:   2026-10-11 (Sun, 11 Oct 2026)

  Changed paths:
    M t/TEST

  Log Message:
  -----------
  Restore name of dump_manifested_tests() to dump_tests()

There's nothing within this subroutine that is specific to
manifested_tests.  If we simply call it dump_tests() we can use it
within t/harness without redefining it.


  Commit: 36bb9710d52d2733a46b0f73204e759d8649d9b5
      
https://github.com/Perl/perl5/commit/36bb9710d52d2733a46b0f73204e759d8649d9b5
  Author: James E Keenan <[email protected]>
  Date:   2026-10-11 (Sun, 11 Oct 2026)

  Changed paths:
    M t/harness

  Log Message:
  -----------
  Uncuddle some else's

Add inline comments where we're pulling in subroutines defined in
t/TEST.

Some other small tidying.


  Commit: e88ae090be200bee2e2960dd92721fa3dce47a1e
      
https://github.com/Perl/perl5/commit/e88ae090be200bee2e2960dd92721fa3dce47a1e
  Author: James E Keenan <[email protected]>
  Date:   2026-10-11 (Sun, 11 Oct 2026)

  Changed paths:
    M t/harness

  Log Message:
  -----------
  Move declaration of variable closer to where used


  Commit: 1dd3bf0b86b931e4da6837368dbd0a6fa4aec04d
      
https://github.com/Perl/perl5/commit/1dd3bf0b86b931e4da6837368dbd0a6fa4aec04d
  Author: James E Keenan <[email protected]>
  Date:   2026-10-11 (Sun, 11 Oct 2026)

  Changed paths:
    M t/harness
    M t/porting/test_testlist.t

  Log Message:
  -----------
  Reactivate (simpler) t/porting/test_testlist.t

At this point, we're comparing the manifested tests found by t/TEST to
match all the tests found by t/harness.  We have not yet implemented
testing of generated tests in t/harness, so we're not yet concerned with
the number or identity of generated tests found by t/TEST.


  Commit: 3c259046bf244c1322a660645088e52ec6402c99
      
https://github.com/Perl/perl5/commit/3c259046bf244c1322a660645088e52ec6402c99
  Author: James E Keenan <[email protected]>
  Date:   2026-10-11 (Sun, 11 Oct 2026)

  Changed paths:
    M t/harness

  Log Message:
  -----------
  t/harness: Locate generated tests and test them


  Commit: 5fa3bc1407d79122601125be3a9ae3da53561710
      
https://github.com/Perl/perl5/commit/5fa3bc1407d79122601125be3a9ae3da53561710
  Author: James E Keenan <[email protected]>
  Date:   2026-10-11 (Sun, 11 Oct 2026)

  Changed paths:
    M t/harness

  Log Message:
  -----------
  Don't mix specific tests with generated tests


  Commit: eee1abfa75f3e56e1c39c3e55acc22983729b602
      
https://github.com/Perl/perl5/commit/eee1abfa75f3e56e1c39c3e55acc22983729b602
  Author: James E Keenan <[email protected]>
  Date:   2026-10-11 (Sun, 11 Oct 2026)

  Changed paths:
    M t/porting/test_testlist.t

  Log Message:
  -----------
  Pause test_testlist.t temporarily


  Commit: 9d8468f7f0568583b0615b3fb7b6f12284bb766a
      
https://github.com/Perl/perl5/commit/9d8468f7f0568583b0615b3fb7b6f12284bb766a
  Author: James E Keenan <[email protected]>
  Date:   2026-10-11 (Sun, 11 Oct 2026)

  Changed paths:
    M t/TEST

  Log Message:
  -----------
  Reposition identification of generated tests


  Commit: 3beedcf6f145aa7f88ab289794236a308a937034
      
https://github.com/Perl/perl5/commit/3beedcf6f145aa7f88ab289794236a308a937034
  Author: James E Keenan <[email protected]>
  Date:   2026-10-11 (Sun, 11 Oct 2026)

  Changed paths:
    M t/harness

  Log Message:
  -----------
  _compute_tests_and_ordering:  Improve encapsulation

%total_time was used only within _compute_tests_and_ordering() and
sort_by_execution_order.  The latter, in turn, was/is only called within
_compute_tests_and_ordering().  Bring declaration of %total_time inside
_compute_tests_and_ordering().  Transform  sort_by_execution_order()
into a lexical sub declared within _compute_tests_and_ordering().


  Commit: b12802470cc9581a4856fc2a2e41c8f3149a6ce7
      
https://github.com/Perl/perl5/commit/b12802470cc9581a4856fc2a2e41c8f3149a6ce7
  Author: James E Keenan <[email protected]>
  Date:   2026-10-11 (Sun, 11 Oct 2026)

  Changed paths:
    M t/porting/test_testlist.t

  Log Message:
  -----------
  test_testlib.t: Add testing subroutines and use them

Add 3 subroutines which can be used to test a given list of test files
via t/TEST, t/harness and ./runtests choose, respectively.

Add 2 subroutines used to create lookup tables within the 3
aforementioned subroutines.

Rearrange the file so that subroutines are defined beneath the test code
and superseded tests are placed in an END block.  (We'll get back to
them later.)


  Commit: dd5ddd947080d9789d90738c1209ded692566b66
      
https://github.com/Perl/perl5/commit/dd5ddd947080d9789d90738c1209ded692566b66
  Author: James E Keenan <[email protected]>
  Date:   2026-10-11 (Sun, 11 Oct 2026)

  Changed paths:
    M t/porting/test_testlist.t

  Log Message:
  -----------
  Test generated tests for dist/Devel-PPPort

Using recently developed testing subroutines.  Add inline comments;
deleted commented-out code.


  Commit: 5ada024e2a2afff30d4051c3e5c760863d8cca6f
      
https://github.com/Perl/perl5/commit/5ada024e2a2afff30d4051c3e5c760863d8cca6f
  Author: James E Keenan <[email protected]>
  Date:   2026-10-11 (Sun, 11 Oct 2026)

  Changed paths:
    M t/porting/test_testlist.t

  Log Message:
  -----------
  test_testlist.t: Restore pre-5.43.4 tests

Suitably tweaked.


  Commit: 1f5ed22b48bb80b6fbd32c1c7f65e4a4ef55c482
      
https://github.com/Perl/perl5/commit/1f5ed22b48bb80b6fbd32c1c7f65e4a4ef55c482
  Author: James E Keenan <[email protected]>
  Date:   2026-10-11 (Sun, 11 Oct 2026)

  Changed paths:
    M t/porting/test_testlist.t

  Log Message:
  -----------
  Skip two instances of 'test_via_runtests'

Experimentation showed that while the test suite ran as expected when invoked
with 'make test', 'make test_harness', etc., we experienced indefinite hangs
during 't/porting/test_testlist.t' whenever we called the following:

    $ make test_porting

    # what's "inside" 'make test_porting':
    $ TEST_ARGS='porting/*.t lib/diagnostics.t' TESTFILE=harness ./runtests 
choose

    $ TEST_ARGS='porting/test_testlist.t' TESTFILE=TEST ./runtests choose

    $ TEST_ARGS='porting/*.t' TESTFILE=TEST ./runtests choose

Examination of the output of 'ps auxwww | grep testlist' showed that
additional processes were being spawned.

But the following did *not* hang:

    $ TESTFILE=harness ./runtests choose

    $ TESTFILE=TEST ./runtests choose

It is not clear why passing "TEST_ARGS=[something which expands to include
porting/test_testlist.t]" to "./runtests choose" behaves in this way.  But as
that is not a way most users would trigger these tests, we can defer further
investigation for now via 'todo_skip'.


  Commit: fc00efc0dcbb4ca9a6ec1b7911d12c5eec97d33b
      
https://github.com/Perl/perl5/commit/fc00efc0dcbb4ca9a6ec1b7911d12c5eec97d33b
  Author: James E Keenan <[email protected]>
  Date:   2026-10-11 (Sun, 11 Oct 2026)

  Changed paths:
    M t/porting/test_testlist.t

  Log Message:
  -----------
  Avoid uninitialized value warning


  Commit: b32d53bd071609a667d86fe376b1df0b44f3b1a9
      
https://github.com/Perl/perl5/commit/b32d53bd071609a667d86fe376b1df0b44f3b1a9
  Author: James E Keenan <[email protected]>
  Date:   2026-10-11 (Sun, 11 Oct 2026)

  Changed paths:
    M t/porting/test_testlist.t

  Log Message:
  -----------
  test_testlist.t:  Skip 4 tests on Win32

We need to determine cause of test failures on that platform.


Compare: https://github.com/Perl/perl5/compare/8ea6a5dd5a76...b32d53bd0716

To unsubscribe from these emails, change your notification settings at 
https://github.com/Perl/perl5/settings/notifications

Reply via email to