driazati opened a new issue, #11567: URL: https://github.com/apache/tvm/issues/11567
These tests were found to be flaky (intermittently failing on `main` or failed in a PR with unrelated changes). See [the docs](https://github.com/apache/tvm/blob/main/docs/contribute/ci.rst#handling-flaky-failures) for details. ### Tests(s) - `tests/python/frontend/tflite/test_forward.py::test_all_unary_elemwise` ### Jenkins Links - https://ci.tlcpack.ai/job/tvm/job/PR-11565/1/display/redirect looks like a problem with tolerances again ``` [2022-06-03T21:07:37.511Z] ___________________________ test_all_unary_elemwise ____________________________ [2022-06-03T21:07:37.511Z] [2022-06-03T21:07:37.511Z] def test_all_unary_elemwise(): [2022-06-03T21:07:37.511Z] _test_forward_unary_elemwise(_test_abs, int_quant_dtype=tf.int8) [2022-06-03T21:07:37.511Z] _test_forward_unary_elemwise(_test_abs, int_quant_dtype=tf.int16) [2022-06-03T21:07:37.511Z] _test_forward_unary_elemwise(_test_floor) [2022-06-03T21:07:37.511Z] _test_forward_unary_elemwise(_test_exp) [2022-06-03T21:07:37.511Z] _test_forward_unary_elemwise(_test_log, negative=False) [2022-06-03T21:07:37.511Z] _test_forward_unary_elemwise(_test_square) [2022-06-03T21:07:37.511Z] _test_forward_unary_elemwise(_test_sin) [2022-06-03T21:07:37.511Z] _test_forward_unary_elemwise(_test_neg) [2022-06-03T21:07:37.511Z] _test_forward_unary_elemwise(_test_sqrt, negative=False) [2022-06-03T21:07:37.511Z] # tensorflow version upgrade support [2022-06-03T21:07:37.511Z] if tf.__version__ < LooseVersion("2.6.1"): [2022-06-03T21:07:37.511Z] _test_forward_unary_elemwise(_test_rsqrt, negative=False, int_quant_dtype=tf.uint8) [2022-06-03T21:07:37.511Z] else: [2022-06-03T21:07:37.511Z] > _test_forward_unary_elemwise(_test_rsqrt, negative=False, int_quant_dtype=tf.int8) [2022-06-03T21:07:37.511Z] [2022-06-03T21:07:37.511Z] tests/python/frontend/tflite/test_forward.py:2130: [2022-06-03T21:07:37.511Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [2022-06-03T21:07:37.511Z] tests/python/frontend/tflite/test_forward.py:2102: in _test_forward_unary_elemwise [2022-06-03T21:07:37.511Z] test_op(data, quantized=True, int_quant_dtype=int_quant_dtype) [2022-06-03T21:07:37.511Z] tests/python/frontend/tflite/test_forward.py:1949: in _test_rsqrt [2022-06-03T21:07:37.511Z] np.squeeze(tvm_output[0]), np.squeeze(tflite_output[0]), rtol=1e-5, atol=1e-2 [2022-06-03T21:07:37.511Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [2022-06-03T21:07:37.511Z] [2022-06-03T21:07:37.511Z] actual = array([ 127, -91, -102, -107, -110, -112], dtype=int8) [2022-06-03T21:07:37.511Z] desired = array([ 127, -91, -102, -107, -110, -111], dtype=int8), rtol = 1e-05 [2022-06-03T21:07:37.511Z] atol = 0.01 [2022-06-03T21:07:37.511Z] [2022-06-03T21:07:37.511Z] def assert_allclose(actual, desired, rtol=1e-7, atol=1e-7): [2022-06-03T21:07:37.511Z] """Version of np.testing.assert_allclose with `atol` and `rtol` fields set [2022-06-03T21:07:37.511Z] in reasonable defaults. [2022-06-03T21:07:37.511Z] [2022-06-03T21:07:37.511Z] Arguments `actual` and `desired` are not interchangeable, since the function [2022-06-03T21:07:37.511Z] compares the `abs(actual-desired)` with `atol+rtol*abs(desired)`. Since we [2022-06-03T21:07:37.511Z] often allow `desired` to be close to zero, we generally want non-zero `atol`. [2022-06-03T21:07:37.511Z] """ [2022-06-03T21:07:37.511Z] actual = np.asanyarray(actual) [2022-06-03T21:07:37.511Z] desired = np.asanyarray(desired) [2022-06-03T21:07:37.511Z] np.testing.assert_allclose(actual.shape, desired.shape) [2022-06-03T21:07:37.511Z] > np.testing.assert_allclose(actual, desired, rtol=rtol, atol=atol, verbose=True) [2022-06-03T21:07:37.511Z] E AssertionError: [2022-06-03T21:07:37.511Z] E Not equal to tolerance rtol=1e-05, atol=0.01 [2022-06-03T21:07:37.511Z] E [2022-06-03T21:07:37.511Z] E Mismatched elements: 1 / 6 (16.7%) [2022-06-03T21:07:37.511Z] E Max absolute difference: 1 [2022-06-03T21:07:37.511Z] E Max relative difference: 0.00900901 [2022-06-03T21:07:37.511Z] E x: array([ 127, -91, -102, -107, -110, -112], dtype=int8) [2022-06-03T21:07:37.511Z] E y: array([ 127, -91, -102, -107, -110, -111], dtype=int8) [2022-06-03T21:07:37.511Z] [2022-06-03T21:07:37.511Z] python/tvm/testing/utils.py:106: AssertionError ``` cc @leandron -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
