On Wed, Jun 24, 2026 at 05:18:47PM +0100, Gavin Smith wrote: > This caused a problem at one point when I was updating the test results. The > condition wasn't met and the test wasn't updated, and then the test > suite failed under different conditions. (I can't remember the exact > circumstances and it was possible that it was the other way around, that > I updated the test results with XS enabled, and then the tests failed with > pure Perl, although I don't believe that would have happened as the test > should have been skipped under pure Perl.)
I do not remember experiencing this issue myself, but I agree that it is likely to be problematic, for instance, if the output changes and the tests with TEXINFO_XS=omit are not run often, the pure Perl only tests would become incorrect. Another issue is that these tests do not test the C code, which is our main target now, and therefore are not very useful. > So tests that are only run under certain conditions cause a problem with > updating the results. Hence if we are to have conditional tests the tests > should be enabled under "normal" circumstances. For example, if we have > tests that require XS conversion, then the results should only be updated > if XS conversion is enabled. As far as I can tell, this is already the case. If a test is skipped, the test function returns and the regeneration of the test results does not happen. > I remember we agreed that the variables like TEXINFO_XS_CONVERT were going to > go away, so as a preliminary step I am going to try to remove these and > replace > them with a single variable (TEXINFO_XS). I think that it is a good idea, however I do not think that it will solve the above issue, as this discrepancy happens with TEXINFO_XS=omit, which we should keep. And the real issue, that these tests are non reproducible in C (for tests wirht transliterations), or not easy to do (for t/html_tests.t spaces_in_line_break_in_verb_w) still remain. -- Pat
