This is an automated email from the git hooks/post-receive script.
git pushed a commit to branch wl/browser-all
in repository enlightenment.
View the commit online.
commit f966904674de038b36a378a33091167c65615626
Author: Cedric BAIL <[email protected]>
AuthorDate: Sun Aug 9 08:38:25 2026 -0600
tests - record expected failures, and notice when they stop failing
The libwayland stride bug is not going to be fixed and rolled out quickly,
so BadBufferTest.client_lies_about_buffer_size will fail for a long time
whatever we do. Leaving E-21 permanently red is the worst option: a filter
that is always red gets ignored, and then a real regression hides behind it.
expected-failures.txt records such tests with the reason and, more usefully,
the condition under which the entry should go. wlcs-task.sh subtracts them
from the failure count and reports them as "xfail", so E-21 now reads
"failed 1 xfail 1" - the one remaining failure being the SIGBUS case, which
really is ours.
The half that earns its keep is the other direction: if a test listed here
starts passing, that is an error, not a quiet success. It means the external
fix has landed and our workaround may no longer be needed. Without that, a
workaround outlives its reason and nobody finds out for years - so an
unexpected pass exits 4 and says what to do about it.
Verified both ways: the known failure is absorbed, and deliberately listing
a passing test is reported as UNEXPECTED PASS rather than ignored.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01FtoiXoSKUmZb6Aix6U3GZS
---
src/tests/wlcs/expected-failures.txt | 14 ++++++++++++++
src/tests/wlcs/wlcs-task.sh | 26 ++++++++++++++++++++++++--
2 files changed, 38 insertions(+), 2 deletions(-)
diff --git a/src/tests/wlcs/expected-failures.txt b/src/tests/wlcs/expected-failures.txt
new file mode 100644
index 000000000..72d5c9c97
--- /dev/null
+++ b/src/tests/wlcs/expected-failures.txt
@@ -0,0 +1,14 @@
+# Tests that fail for reasons outside this tree.
+#
+# <full gtest test name> <one-line reason>
+#
+# wlcs-task.sh subtracts these from the failure count and reports them as
+# "xfail". Crucially it also fails if one of them *passes*: an expected failure
+# that starts working is the signal that the external fix has landed and our
+# workaround can go. Without that, a workaround outlives its reason and nobody
+# finds out for years.
+#
+# Keep the reason specific enough that someone can check whether it still
+# holds, and delete the line rather than the check when it stops holding.
+
+BadBufferTest.client_lies_about_buffer_size libwayland wl_shm accepts a stride too small for the format; see libwayland-shm-stride-bug.md. E rejects the buffer at attach so the out-of-bounds read is closed, but the test asserts the error on wl_shm_pool from create_buffer, which is libwayland's request and not ours. Remove when libwayland validates stride against bytes-per-pixel.
diff --git a/src/tests/wlcs/wlcs-task.sh b/src/tests/wlcs/wlcs-task.sh
index 115995757..e99a6c5a2 100755
--- a/src/tests/wlcs/wlcs-task.sh
+++ b/src/tests/wlcs/wlcs-task.sh
@@ -22,6 +22,7 @@ set -eu
here=$(cd "$(dirname "$0")" && pwd)
tasks="$here/tasks.txt"
+xfails="$here/expected-failures.txt"
WLCS=${WLCS:-}
if [ -z "$WLCS" ]; then
@@ -60,6 +61,27 @@ run_one() {
return 3
fi
+ # Split the failures into "ours" and "known to be someone else's".
+ xfail=0; xpass=0
+ if [ -f "$xfails" ]; then
+ while read -r name _rest; do
+ case "$name" in ''|\#*) continue ;; esac
+ if printf '%s\n' "$out" | grep -qE "^\[ FAILED \] $(echo "$name" | sed 's/[.[\*^$]/\\&/g') "; then
+ xfail=$((xfail + 1)); bad=$((bad - 1))
+ elif printf '%s\n' "$out" | grep -qE "^\[ OK \] $(echo "$name" | sed 's/[.[\*^$]/\\&/g') "; then
+ xpass=$((xpass + 1))
+ echo "$task: $name now PASSES but is listed as an expected failure." >&2
+ echo "wlcs-task: the external reason has gone away - drop the entry, and check whether our workaround is still needed." >&2
+ fi
+ done < "$xfails"
+ fi
+
+ if [ "$xpass" -gt 0 ]; then
+ printf '%-8s passed %-5s failed %-5s skipped %-5s xfail %-5s UNEXPECTED PASS\n' \
+ "$task" "$ok" "$bad" "$skip" "$xfail"
+ return 4
+ fi
+
if [ "$bad" -eq 0 ] && [ "$ok" -eq 0 ]; then
state='not implemented'
elif [ "$bad" -eq 0 ]; then
@@ -68,8 +90,8 @@ run_one() {
state='not yet'
fi
- printf '%-8s passed %-5s failed %-5s skipped %-5s %s\n' \
- "$task" "$ok" "$bad" "$skip" "$state"
+ printf '%-8s passed %-5s failed %-5s skipped %-5s xfail %-5s %s\n' \
+ "$task" "$ok" "$bad" "$skip" "$xfail" "$state"
[ "$bad" -eq 0 ] && [ "$ok" -gt 0 ]
}
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
