On Thu, 2025-02-27 at 12:00 +0100, Miro Hrončok wrote: > Hello PyPy folks. > > I have recently discovered that in Fedora, when building PyPy, we run > test.regrtest but we don't make test failures terminate the build. > > For a couple times, I've reported test hangs because that was unfortunately > the > only problem that we were able to spot. > > Tests that run but don't fail the build are kinda useless. I want to remedy > that but I noticed that many tests fail for quite obvious reasons, for > example: > > ====================================================================== > FAIL: test_getsitepackages (test.test_site.HelperFunctionsTests) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File > "/builddir/build/BUILD/pypy3.10-v7.3.19-src/lib-python/3/test/test_site.py", > line 335, in test_getsitepackages > self.assertEqual(dirs[0], wanted) > AssertionError: 'xoxo/lib64/pypy3.10/site-packages' != > 'xoxo/lib64/python3.10/site-packages' > - xoxo/lib64/pypy3.10/site-packages > ? ^^ > + xoxo/lib64/python3.10/site-packages > ? ^^^^ > ---------------------------------------------------------------------- > > > I have examined the GitHub actions configuration in pypy/pypy but I don't see > the tests run. > > What are the expectations about the test results? Should we check that the > failure rate does not exceed some percentage? Or should we skip know failures > one by one? Or report them to you? > > Or do we keep running the tests just that we could see the failure in the > logs > but don't fail the build for that? > > How do you use those tests in PyPy?
Others can probably tell you more, but PyPy's running the test suite via pytest. As for failing, yes, some are expected to fail. See the buildbots [1] for the current status. [1] https://buildbot.pypy.org/ -- Best regards, Michał Górny
signature.asc
Description: This is a digitally signed message part
_______________________________________________ pypy-dev mailing list -- pypy-dev@python.org To unsubscribe send an email to pypy-dev-le...@python.org https://mail.python.org/mailman3/lists/pypy-dev.python.org/ Member address: arch...@mail-archive.com
