On 27/02/2025 22:05, Michał Górny wrote:
On Thu, 2025-02-27 at 12:00 +0100, Miro Hrončok wrote:
Hello PyPy folks.
I have recently discovered that in Fedora, when building PyPy, we run
test.regrtest but we don't make test failures terminate the build.
...
I have examined the GitHub actions configuration in pypy/pypy but I don't see
the tests run.
What are the expectations about the test results? Should we check that the
failure rate does not exceed some percentage? Or should we skip know failures
one by one? Or report them to you?
Or do we keep running the tests just that we could see the failure in the logs
but don't fail the build for that?
How do you use those tests in PyPy?
Others can probably tell you more, but PyPy's running the test suite via
pytest. As for failing, yes, some are expected to fail. See the
buildbots [1] for the current status.
[1] https://buildbot.pypy.org/
We don't expect all the tests to pass. I guess we should mark the
failing stdlib tests with a unittest.expectedFailure decorator
specialized for PyPy and optionally open issues for the failing tests.
In the main time, I would recommend distros do what we did in the
conda-forge recipe and do some minimal testing [0] including test_ssl
(which is particulary important to make sure pip can work) and some
sanity checking.
Matti
[0]
https://github.com/conda-forge/pypy3.6-feedstock/blob/f44727fc07bc1ebaa4849c0b712087120e85be1e/recipe/meta.yaml#L94
_______________________________________________
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com
