sadpandajoe commented on code in PR #36196:
URL: https://github.com/apache/superset/pull/36196#discussion_r2583886900
##########
superset-frontend/playwright/tests/experimental/README.md:
##########
@@ -19,52 +19,98 @@ under the License.
# Experimental Playwright Tests
-This directory contains Playwright tests that are still under development or
validation.
-
## Purpose
-Tests in this directory run in "shadow mode" with `continue-on-error: true` in
CI:
-- Failures do NOT block PR merges
-- Allows tests to run in CI to validate stability before promotion
-- Provides visibility into test reliability over time
+This directory contains **experimental** Playwright E2E tests that are being
developed and stabilized before becoming part of the required test suite.
+
+## How Experimental Tests Work
+
+### Running Tests
+
+**By default (CI and local), experimental tests are EXCLUDED:**
+```bash
+npm run playwright:test
+# Only runs stable tests (tests/auth/*)
+```
+
+**To include experimental tests, set the environment variable:**
+```bash
+INCLUDE_EXPERIMENTAL=true npm run playwright:test
+# Runs all tests including experimental/
+```
+
+### CI Behavior
+
+- **Required CI jobs**: Experimental tests are excluded by default
+ - Tests in `experimental/` do NOT block merges
+ - Failures in `experimental/` do NOT fail the build
-## Promoting Tests to Stable
+- **Experimental CI jobs** (optional): Use `TEST_PATH=experimental/`
+ - `.github/workflows/bashlib.sh` sets `INCLUDE_EXPERIMENTAL=true` when
`TEST_PATH` is provided
Review Comment:
The README is correct - bashlib.sh does implement this behavior at lines
264-265:
```bash
if [ -n "$TEST_PATH" ]; then
...
export INCLUDE_EXPERIMENTAL=true
npx playwright test "${TEST_PATH}" ...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
