This is an automated email from the ASF dual-hosted git repository.
masahi pushed a commit to branch unity
in repository https://gitbox.apache.org/repos/asf/tvm.git
The following commit(s) were added to refs/heads/unity by this push:
new 6f2fe457c2 [Unity][UnitTest] Increase atol to resolve flaky CI failure
(#16340)
6f2fe457c2 is described below
commit 6f2fe457c285a2617c1f3f6aea368b9a027dcbbb
Author: Eric Lunderberg <[email protected]>
AuthorDate: Wed Jan 3 13:19:33 2024 -0600
[Unity][UnitTest] Increase atol to resolve flaky CI failure (#16340)
The `tests/python/relax/test_frontend_onnx.py::test_attention` unit
test currently has sporadic failures (5/200 executions), which can
cause failures for unrelated changes
(e.g. [PR#16304](https://github.com/apache/tvm/pull/16304), [CI
link](https://ci.tlcpack.ai/blue/organizations/jenkins/tvm-unity/detail/PR-16304/3/pipeline)).
This commit specifies a tolerance to use for comparisons, to avoid
these spurious CI failures.
---
tests/python/relax/test_frontend_onnx.py | 2 ++
1 file changed, 2 insertions(+)
diff --git a/tests/python/relax/test_frontend_onnx.py
b/tests/python/relax/test_frontend_onnx.py
index 19be3e0b03..748119f6f9 100644
--- a/tests/python/relax/test_frontend_onnx.py
+++ b/tests/python/relax/test_frontend_onnx.py
@@ -1226,6 +1226,8 @@ def test_attention(dynamic):
"mask_index": mask_index,
"relative_position_bias": relative_position_bias,
},
+ # Maximum observed delta from 500 iterations was 2e-4.
+ atol=1e-3,
)
# "present" output should be nullptr when the "past" input isn't
included,
# but ort requires an output shape to be specified?
