janhoy commented on PR #4738: URL: https://github.com/apache/solr/pull/4738#issuecomment-5370881131
I've now made the whole suite opt-in, which I think addresses the stability/maturity concerns raised here: - The suites carry a new `@SeleniumTest` test group annotation (disabled by default, same mechanics as `@Nightly`/`@Monster` but its own group), enabled with `-Ptests.selenium=true`. Without the flag all 53 tests group-skip in a fraction of a second — no cluster, no browser, no node — so they can never block or slow down regular development. - A missing Chrome now *fails* the tests instead of skipping: if you explicitly opted in, a silent skip would be a false green. - A new GitHub Actions workflow (`admin-ui-test.yml`) runs them on PRs that touch the webapp, the v2 API contract (`solr/api`) or the v2 API implementations (`handler/admin/api`) — exactly the places a UI regression can originate, per Eric's point that v2 migration is what's raising the risk. On nightly (@dsmiley): I deliberately didn't group them into `@Nightly` — ASF Jenkins has no Chrome so they'd never run there anyway, and per Eric few people watch the nightly results. The targeted PR workflow gives faster and more visible feedback. That said, any CI environment with Chrome can add `-Ptests.selenium=true` to a nightly job later if we want both. Docs in `dev-docs/admin-ui-tests.md` and `dev-docs/gradle-help/tests.txt` (`gradlew helpTests`). Also renamed the browser override to `-Dtests.selenium.chrome.binary` to match the flag. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
