[tvm] branch main updated: [AOT] Rerun FVP test incase of first attempt failure (#10408)

[masahi]

This is an automated email from the ASF dual-hosted git repository.

masahi pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tvm.git


The following commit(s) were added to refs/heads/main by this push:
     new d7ccb91  [AOT] Rerun FVP test incase of first attempt failure (#10408)
d7ccb91 is described below

commit d7ccb91d16e7374ae651ca27f376c339f6e04e3f
Author: lhutton1 <luke.hut...@arm.com>
AuthorDate: Sun Mar 6 23:10:15 2022 +0000

    [AOT] Rerun FVP test incase of first attempt failure (#10408)
    
    * [AOT] Rerun FVP test incase of first attempt failure
    
    In light of the flaky failures seen when running these sorts of tests
    (issues: 10300 and 10314), adding a second attempt if the first attempt
    fails to try to help reduce the number of failures observed in CI.
    
    Change-Id: I25eb268555baab85fc7e70d2d70a37f3be49a54b
    
    * apply same logic to CMSISNN tests as well
    
    Change-Id: I0afb54676e83af06fcd9ab2cc1fc2e87002031cd
    
    * rebase and print errors to stderr to capture in pytest log
    
    Change-Id: Ibedff4515050421d98462bfbd95d5cdf77b12412
    
    * reduce scope of retry
    
    Reduces the scope of the retrial to only the portion that runs the
    test module on the FVP.
    
    Change-Id: I4520f50aea5cc8e28c03b49da18612cc7f8b8045
---
 tests/python/relay/aot/aot_test_utils.py | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/tests/python/relay/aot/aot_test_utils.py 
b/tests/python/relay/aot/aot_test_utils.py
index 5ca25fd..9a62e2e 100644
--- a/tests/python/relay/aot/aot_test_utils.py
+++ b/tests/python/relay/aot/aot_test_utils.py
@@ -15,6 +15,7 @@
 # specific language governing permissions and limitations
 # under the License.
 
+import sys
 import datetime
 import itertools
 import json
@@ -843,7 +844,14 @@ def run_and_check(
     if verbose:
         print("Run command:\n", run_command)
 
-    subprocess_check_log_output(run_command, build_path, run_log_path)
+    # TODO(lhutton1) This is a quick and dirty work around to help temporarily 
reduce
+    # the flakyness of the tests. Will remove once #10300 and #10314 are 
resolved.
+    try:
+        subprocess_check_log_output(run_command, build_path, run_log_path)
+    except RuntimeError as err:
+        print("Failed to run the module, having a second attempt...", 
file=sys.stderr)
+        print(err, file=sys.stderr)
+        subprocess_check_log_output(run_command, build_path, run_log_path)
 
     with open(run_log_path) as run_log:
         assert AOT_SUCCESS_TOKEN in run_log.read()