leezu opened a new issue #19255: URL: https://github.com/apache/incubator-mxnet/issues/19255
## Description For skipped tests, we will see the output `../tests/python/unittest/test_gluon.py::test_hybrid_static_memory[True-False] [WARNING] Error seen with seeded test, use MXNET_TEST_SEED=458439587 to reproduce.` implying an error was seen, even though the skipping of the test was intentional. This is due to the very broad exception handling in https://github.com/apache/incubator-mxnet/blob/7dcfedca704f39b4b9b7497dabf3fea47ad40df4/tests/python/unittest/common.py#L225-L235 as skipping a test in pytest will raise the `Skipped` `BaseException`, which the `with_seed` decorator interprets as an error https://github.com/pytest-dev/pytest/blob/303030c14130a5777bdaace678b9f4adb07416ab/src/_pytest/outcomes.py#L51-L63 Full output ``` % python3 -m pytest -s --verbose ../tests/python/unittest/test_gluon.py::test_hybrid_static_memory ~/src/mxnet-master/build testscleanup ip-172-31-95-96 ================================================================================= test session starts ================================================================================= platform linux -- Python 3.8.5, pytest-6.0.2, py-1.9.0, pluggy-0.13.1 -- /home/ubuntu/.pyenv/versions/3.8.5/bin/python3 cachedir: .pytest_cache rootdir: /home/ubuntu/src/mxnet-master, configfile: pytest.ini plugins: timeout-1.4.2, env-0.6.2 timeout: 1200.0s timeout method: signal timeout func_only: False collected 4 items ../tests/python/unittest/test_gluon.py::test_hybrid_static_memory[False-False] Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=1639191096 to reproduce. [23:00:40] ../src/storage/storage.cc:199: Using Pooled (Naive) StorageManager for CPU PASSED ../tests/python/unittest/test_gluon.py::test_hybrid_static_memory[False-True] PASSED ../tests/python/unittest/test_gluon.py::test_hybrid_static_memory[True-False] [WARNING] Error seen with seeded test, use MXNET_TEST_SEED=458439587 to reproduce. SKIPPED ../tests/python/unittest/test_gluon.py::test_hybrid_static_memory[True-True] PASSED ============================================================================ 3 passed, 1 skipped in 1.56s ============================================================================= ``` One approach to solve the issue, is to remove the `@with_seed` decorator from `common.py` and just rely on the pytest seed marker. (This requires further investigation for cases where `@with_seed` isn't applied to test functions, but applied to inline functions inside test functions; for example in the test large tensor files https://github.com/apache/incubator-mxnet/blob/97d4ba5a133f93ff6075dcde3ef842b23d498a12/tests/nightly/test_large_vector.py#L137-L157) https://github.com/apache/incubator-mxnet/blob/7dcfedca704f39b4b9b7497dabf3fea47ad40df4/conftest.py#L133-L162 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
