Hello Nikos Nikoleris,

I'd like you to do a code review. Please visit

    https://gem5-review.googlesource.com/3962

to review the following change.


Change subject: tests: Don't treat new stats as a cause for failures
......................................................................

tests: Don't treat new stats as a cause for failures

We currently fail the stat diff stage of tests if there are new
stats. This is usually undesirable since this would require any change
that adds a stat to also update the regressions.

Change-Id: Ieadebac6fd17534e1b49b6b9a1d56f037a423325
Signed-off-by: Andreas Sandberg <[email protected]>
Reviewed-by: Nikos Nikoleris <[email protected]>
---
M tests/diff-out
M tests/testing/units.py
2 files changed, 12 insertions(+), 4 deletions(-)



diff --git a/tests/diff-out b/tests/diff-out
index c00d4f3..1eeac81 100755
--- a/tests/diff-out
+++ b/tests/diff-out
@@ -361,8 +361,12 @@
 }

 cleanup();
-# Exit code is 0 if all stats are found (with no extras) & no stats error, 1 otherwise -$status = ($missing_stats == 0 && $added_stats == 0 && $max_err_mag == 0.0) ? 0 : 1;
+# Exit codes:
+# 0 if all stats are found (with no extras) & no stats error
+# 1 if there are additional stats, but no stat errors
+# 2 otherwise
+$no_hard_errors = $missing_stats == 0 && $max_err_mag == 0.0;
+$status = $no_hard_errors ? ($added_stats == 0 ? 0 : 1) : 2;
 exit $status;

 sub cleanup
diff --git a/tests/testing/units.py b/tests/testing/units.py
index e8b87a0..220cf61 100644
--- a/tests/testing/units.py
+++ b/tests/testing/units.py
@@ -270,6 +270,10 @@
         self.stat_diff = os.path.join(_test_base, "diff-out")

     def _run(self):
+        STATUS_OK = 0
+        STATUS_NEW_STATS = 1
+        STATUS_FAILED = 2
+
         stats = "stats.txt"

         cmd = [
@@ -281,9 +285,9 @@
                            stderr=subprocess.PIPE) as p:
             status, stdout, stderr = p.call()

-        if status == 0:
+        if status in (STATUS_OK, STATUS_NEW_STATS):
             return self.ok(stdout=stdout, stderr=stderr)
-        if status == 1:
+        elif status == STATUS_FAILED:
             return self.failure("Statistics mismatch",
                                 stdout=stdout, stderr=stderr)
         else:

--
To view, visit https://gem5-review.googlesource.com/3962
To unsubscribe, visit https://gem5-review.googlesource.com/settings

Gerrit-Project: public/gem5
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ieadebac6fd17534e1b49b6b9a1d56f037a423325
Gerrit-Change-Number: 3962
Gerrit-PatchSet: 1
Gerrit-Owner: Andreas Sandberg <[email protected]>
Gerrit-Reviewer: Nikos Nikoleris <[email protected]>
_______________________________________________
gem5-dev mailing list
[email protected]
http://m5sim.org/mailman/listinfo/gem5-dev

Reply via email to