epugh commented on code in PR #4738: URL: https://github.com/apache/solr/pull/4738#discussion_r3780264506
########## dev-docs/admin-ui-tests.md: ########## @@ -0,0 +1,226 @@ +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +# Admin UI (AngularJS) Browser Test Plan + +This document tracks browser-based test coverage of the old AngularJS Admin UI +(`solr/webapp/web/`), driven by Selenium WebDriver with headless Chrome. + +## How the tests work + +- Tests live in `solr/webapp/src/test/org/apache/solr/webapp/` and extend + `AdminUiTestBase`, which starts a 2-node `MiniSolrCloudCluster` whose Jetty + nodes also serve the Admin UI (opt-in `JettyConfig.Builder#enableAdminUi`), + then starts a headless Chrome via Selenium WebDriver. +- A locally installed Chrome/Chromium is required; without one, tests skip via + `Assume`. Override discovery with `-Dtests.ui.chrome.binary=/path/to/chrome`. + The matching chromedriver is provisioned (and cached) by Selenium Manager. +- Display assertions compare UI text against live JSON from the same node's + admin APIs — never hardcoded values. +- Tests are grouped per screen/feature, so each screen's display and write + tests live in the same class. +- Most tests run against a 2-node cloud cluster; `AdminUiStandaloneTestBase` + additionally supports standalone (user-managed, no ZooKeeper) nodes, whose + UI differs (no Cloud/Collections/Schema Designer menus; the per-core menu + offers query/replication etc. directly). +- On failure, a screenshot, the page source and the browser console log are + saved into the test temp dir. +- Run with: `./gradlew :solr:webapp:test` + +## Coverage by screen Review Comment: this docs, while nice to see in hte PR< I don't think need to stay long term? I wish you coul dlabel content "Here for hte life of the PR, but once merged to main it goes away"! ########## dev-docs/admin-ui-tests.md: ########## @@ -0,0 +1,177 @@ +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +# Admin UI (AngularJS) Browser Test Plan + +This document tracks browser-based test coverage of the old AngularJS Admin UI +(`solr/webapp/web/`), driven by Selenium WebDriver with headless Chrome. + +## How the tests work + +- Tests live in `solr/webapp/src/test/org/apache/solr/webapp/` and extend + `AdminUiTestBase`, which starts a 2-node `MiniSolrCloudCluster` whose Jetty + nodes also serve the Admin UI (opt-in `JettyConfig.Builder#enableAdminUi`), + then starts a headless Chrome via Selenium WebDriver. +- A locally installed Chrome/Chromium is required; without one, tests skip via + `Assume`. Override discovery with `-Dtests.ui.chrome.binary=/path/to/chrome`. + The matching chromedriver is provisioned (and cached) by Selenium Manager. +- Display assertions compare UI text against live JSON from the same node's + admin APIs — never hardcoded values. +- Tests are grouped per screen/feature, so each screen's display and write + tests live in the same class. +- Most tests run against a 2-node cloud cluster; `AdminUiStandaloneTestBase` + additionally supports standalone (user-managed, no ZooKeeper) nodes, whose + UI differs (no Cloud/Collections/Schema Designer menus; the per-core menu + offers query/replication etc. directly). +- On failure, a screenshot, the page source and the browser console log are + saved into the test temp dir. +- Run with: `./gradlew :solr:webapp:test` + +## Coverage by screen Review Comment: I don't. thinks this is useful... this just looks like a lot of content that isn't helpful to me, and will be out of date. Now, if we think AI needs this to understand the tests, but I don't think that either... Also, i wondered if this was a checked in "lets work our way through all the screens", but since everythign has a checked checkbox, i dont' think it's that either. ########## solr/webapp/src/test-files/log4j2.xml: ########## @@ -0,0 +1,40 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> +<!-- We're configuring testing to be synchronous due to "logging polution", see SOLR-13268 --> Review Comment: typo in the word polution.... ########## dev-docs/admin-ui-tests.md: ########## @@ -0,0 +1,226 @@ +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +# Admin UI (AngularJS) Browser Test Plan + +This document tracks browser-based test coverage of the old AngularJS Admin UI +(`solr/webapp/web/`), driven by Selenium WebDriver with headless Chrome. + +## How the tests work + +- Tests live in `solr/webapp/src/test/org/apache/solr/webapp/` and extend + `AdminUiTestBase`, which starts a 2-node `MiniSolrCloudCluster` whose Jetty + nodes also serve the Admin UI (opt-in `JettyConfig.Builder#enableAdminUi`), + then starts a headless Chrome via Selenium WebDriver. +- A locally installed Chrome/Chromium is required; without one, tests skip via + `Assume`. Override discovery with `-Dtests.ui.chrome.binary=/path/to/chrome`. + The matching chromedriver is provisioned (and cached) by Selenium Manager. +- Display assertions compare UI text against live JSON from the same node's + admin APIs — never hardcoded values. +- Tests are grouped per screen/feature, so each screen's display and write + tests live in the same class. +- Most tests run against a 2-node cloud cluster; `AdminUiStandaloneTestBase` + additionally supports standalone (user-managed, no ZooKeeper) nodes, whose + UI differs (no Cloud/Collections/Schema Designer menus; the per-core menu + offers query/replication etc. directly). +- On failure, a screenshot, the page source and the browser console log are + saved into the test temp dir. +- Run with: `./gradlew :solr:webapp:test` + +## Coverage by screen + +### Smoke navigation — `AdminUiSmokeTest` +- [x] Every node-level route (`/`, `~logging`, `~logging/level`, + `~cloud?view=nodes|tree|zkstatus|graph`, `~cores`, `~collections`, + `~schema-designer`, `~security`, `~java-properties`, `~threads`, `login`), + every per-collection route and the per-core routes render their main + content element without severe browser console errors. + +### Dashboard — `AdminUiDashboardTest` +- [x] Versions, JVM info and memory bars vs `/admin/info/system`; security + warning when security is disabled. + +### Node-level screens — `AdminUiNodeScreensTest` +- [x] Java Properties: `java.version` value matches `/admin/info/properties` +- [x] Thread Dump: thread list non-empty, Jetty worker thread shown +- [x] Cloud > Nodes: one row per live node, ports match the cluster +- [x] Cloud > Tree: `live_nodes` and `collections` znodes shown +- [x] Cloud > Graph: d3 SVG renders circles for collection/shard/replicas +- [x] Cloud > ZK Status: status green, ensemble size shown +- [x] Security screen: "not enabled" warning without auth +- [x] Login screen: authentication info page without auth + +### Collections screen — `AdminUiCollectionsScreenTest` +- [x] Collection detail display (shards) +- [x] Create + delete collection via the dialogs (verified via API) +- [x] Create + delete alias via the dialogs (verified via LISTALIASES) +- [x] Add + delete replica via the shard detail (verified via cluster state) +- [x] Reload collection (verified via core start time reset) + +### Query screen — `AdminUiQueryScreenTest` +- [x] `*:*` and `id:` queries via the form, `numFound` in the response +- [x] `rows` and `fl` parameters affect the returned documents +- [x] Paramsets dropdown applies a paramset created via the API +- [x] defType dismax/edismax toggles reveal their parameter fields; edismax + query with `qf` returns the expected results +- [x] Raw query parameters (e.g. an extra `fq`) are applied + +### Documents screen — `AdminUiDocumentsScreenTest` +- [x] Form renders with JSON/XML/CSV document types +- [x] Index a JSON document via the form; becomes searchable + +### Schema screen — `AdminUiSchemaScreenTest` +- [x] Field list browsable, editable-schema action buttons shown +- [x] Field detail shows flags matching `/schema/fields`; term info loads +- [x] Add + delete a field via the dialogs (verified via `/schema/fields`) + +### Paramsets screen — `AdminUiParamsetsScreenTest` +- [x] Form renders +- [x] Create + delete a paramset via the form (verified via `/config/params`) + +### Logging screens — `AdminUiLoggingScreenTest` +- [x] Logger level tree renders +- [x] Set + unset a logger level via the tree (verified via API) +- [x] Events viewer shows a WARN event logged in the server JVM (skips itself + when the node's log watcher is blind due to shared-JVM log4j state, see + Known limitations) + +### Core Admin screen — `AdminUiCoreAdminScreenTest` (cloud) and +### `AdminUiCoreAdminStandaloneTest` (standalone) +- [x] Hosted core listed, matching `/admin/cores` +- [x] Reload core via the button (success indicator) +- [x] Standalone-mode menu differences (no cloud menus; core menu offers + query/replication) +- [x] Add core via the dialog (pre-created instance dir) +- [x] Rename core via the dialog +- [x] Swap cores via the dialog (verified by the indexes exchanging) +- [x] Unload core, accepting the native confirm dialog + +### Per-collection display screens — `AdminUiCollectionScreensTest` +- [x] Analysis: `text_general` tokenizes and lowercases entered text +- [x] Files: tree lists `solrconfig.xml`, file content renders +- [x] Segments: at least one segment rendered after commit +- [x] Plugins/Stats: searcher stats present (needs `metricsEnabled=true`) +- [x] Collection overview: shard info; Core overview: numDocs + +### Stream screen — `AdminUiStreamScreenTest` +- [x] A `search(...)` streaming expression executes and renders all docs + +### SQL screen — `AdminUiSqlScreenTest` +- [x] A SQL query executes through the form and the result grid lists the + documents (the sql module is a test-only dependency of `solr:webapp`) + +### Replication screen — `AdminUiReplicationScreenTest` (cloud) and +### `AdminUiReplicationStandaloneTest` (standalone leader/follower) +- [x] Renders index version info in cloud mode +- [x] Follower screen shows the leader's info +- [x] Disable polling, index on the leader, replicate-now transfers the + index, re-enable polling — verified via the replication API + +### Security with BasicAuth — `AdminUiSecurityAuthTest` +- [x] Unauthenticated visit redirects to login; login form authenticates +- [x] Security screen shows authn/authz plugins, users, roles, permissions +- [x] Add a user through the dialog (verified via `/admin/authentication`) +- [x] Add a role for the user through the dialog (verified via API) +- [x] Grant a predefined permission to the role (verified via API) + +### Schema Designer — `AdminUiSchemaDesignerTest` (`@AwaitsFix`) +- [x] Create a new schema, paste a sample doc, analyze; derived field shown — + but the designer backend is too flaky under automation (see Possible UI + bugs), so the test awaits a fix before running by default + +## Deliberately skipped (effort vs value) + +- **JWT/OAuth login flows**: require an external identity provider or heavy + mocking; BasicAuth covers the UI's login/session mechanics. +- **Keystroke-level entry in the security dialogs**: native clicks/keystrokes + into the absolutely-positioned dialogs proved unreliable in headless Chrome; + the dialogs are driven via the Angular controller scope instead. Keyboard + entry is covered by the login form and the other screens' forms. + +## Possible UI bugs to investigate + +Issues surfaced by these tests that look like real bugs, weaknesses or +flakiness in the Admin UI (or its backing APIs) rather than bad test code. +Tests work around them as noted; each deserves investigation and possibly a +JIRA: + +1. **Menu TypeError on per-collection pages**: navigating to any + per-collection screen intermittently logs + `TypeError: Cannot read properties of null (reading 'name')` from + `$scope.showCore` in `js/angular/app.js` — the core selector fires its + change handler with a null core while the menu resolves. Workaround: the + console-error assertion filters this signature. +2. **Collections screen dies without the js-client bundle**: the Review Comment: This specific bug doens't seem too important.... ########## dev-docs/admin-ui-tests.md: ########## @@ -0,0 +1,226 @@ +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +# Admin UI (AngularJS) Browser Test Plan + +This document tracks browser-based test coverage of the old AngularJS Admin UI +(`solr/webapp/web/`), driven by Selenium WebDriver with headless Chrome. + +## How the tests work + +- Tests live in `solr/webapp/src/test/org/apache/solr/webapp/` and extend + `AdminUiTestBase`, which starts a 2-node `MiniSolrCloudCluster` whose Jetty + nodes also serve the Admin UI (opt-in `JettyConfig.Builder#enableAdminUi`), + then starts a headless Chrome via Selenium WebDriver. +- A locally installed Chrome/Chromium is required; without one, tests skip via + `Assume`. Override discovery with `-Dtests.ui.chrome.binary=/path/to/chrome`. + The matching chromedriver is provisioned (and cached) by Selenium Manager. +- Display assertions compare UI text against live JSON from the same node's + admin APIs — never hardcoded values. +- Tests are grouped per screen/feature, so each screen's display and write + tests live in the same class. +- Most tests run against a 2-node cloud cluster; `AdminUiStandaloneTestBase` + additionally supports standalone (user-managed, no ZooKeeper) nodes, whose + UI differs (no Cloud/Collections/Schema Designer menus; the per-core menu + offers query/replication etc. directly). +- On failure, a screenshot, the page source and the browser console log are + saved into the test temp dir. +- Run with: `./gradlew :solr:webapp:test` + +## Coverage by screen + +### Smoke navigation — `AdminUiSmokeTest` +- [x] Every node-level route (`/`, `~logging`, `~logging/level`, + `~cloud?view=nodes|tree|zkstatus|graph`, `~cores`, `~collections`, + `~schema-designer`, `~security`, `~java-properties`, `~threads`, `login`), + every per-collection route and the per-core routes render their main + content element without severe browser console errors. + +### Dashboard — `AdminUiDashboardTest` +- [x] Versions, JVM info and memory bars vs `/admin/info/system`; security + warning when security is disabled. + +### Node-level screens — `AdminUiNodeScreensTest` +- [x] Java Properties: `java.version` value matches `/admin/info/properties` +- [x] Thread Dump: thread list non-empty, Jetty worker thread shown +- [x] Cloud > Nodes: one row per live node, ports match the cluster +- [x] Cloud > Tree: `live_nodes` and `collections` znodes shown +- [x] Cloud > Graph: d3 SVG renders circles for collection/shard/replicas +- [x] Cloud > ZK Status: status green, ensemble size shown +- [x] Security screen: "not enabled" warning without auth +- [x] Login screen: authentication info page without auth + +### Collections screen — `AdminUiCollectionsScreenTest` +- [x] Collection detail display (shards) +- [x] Create + delete collection via the dialogs (verified via API) +- [x] Create + delete alias via the dialogs (verified via LISTALIASES) +- [x] Add + delete replica via the shard detail (verified via cluster state) +- [x] Reload collection (verified via core start time reset) + +### Query screen — `AdminUiQueryScreenTest` +- [x] `*:*` and `id:` queries via the form, `numFound` in the response +- [x] `rows` and `fl` parameters affect the returned documents +- [x] Paramsets dropdown applies a paramset created via the API +- [x] defType dismax/edismax toggles reveal their parameter fields; edismax + query with `qf` returns the expected results +- [x] Raw query parameters (e.g. an extra `fq`) are applied + +### Documents screen — `AdminUiDocumentsScreenTest` +- [x] Form renders with JSON/XML/CSV document types +- [x] Index a JSON document via the form; becomes searchable + +### Schema screen — `AdminUiSchemaScreenTest` +- [x] Field list browsable, editable-schema action buttons shown +- [x] Field detail shows flags matching `/schema/fields`; term info loads +- [x] Add + delete a field via the dialogs (verified via `/schema/fields`) + +### Paramsets screen — `AdminUiParamsetsScreenTest` +- [x] Form renders +- [x] Create + delete a paramset via the form (verified via `/config/params`) + +### Logging screens — `AdminUiLoggingScreenTest` +- [x] Logger level tree renders +- [x] Set + unset a logger level via the tree (verified via API) +- [x] Events viewer shows a WARN event logged in the server JVM (skips itself + when the node's log watcher is blind due to shared-JVM log4j state, see + Known limitations) + +### Core Admin screen — `AdminUiCoreAdminScreenTest` (cloud) and +### `AdminUiCoreAdminStandaloneTest` (standalone) +- [x] Hosted core listed, matching `/admin/cores` +- [x] Reload core via the button (success indicator) +- [x] Standalone-mode menu differences (no cloud menus; core menu offers + query/replication) +- [x] Add core via the dialog (pre-created instance dir) +- [x] Rename core via the dialog +- [x] Swap cores via the dialog (verified by the indexes exchanging) +- [x] Unload core, accepting the native confirm dialog + +### Per-collection display screens — `AdminUiCollectionScreensTest` +- [x] Analysis: `text_general` tokenizes and lowercases entered text +- [x] Files: tree lists `solrconfig.xml`, file content renders +- [x] Segments: at least one segment rendered after commit +- [x] Plugins/Stats: searcher stats present (needs `metricsEnabled=true`) +- [x] Collection overview: shard info; Core overview: numDocs + +### Stream screen — `AdminUiStreamScreenTest` +- [x] A `search(...)` streaming expression executes and renders all docs + +### SQL screen — `AdminUiSqlScreenTest` +- [x] A SQL query executes through the form and the result grid lists the + documents (the sql module is a test-only dependency of `solr:webapp`) + +### Replication screen — `AdminUiReplicationScreenTest` (cloud) and +### `AdminUiReplicationStandaloneTest` (standalone leader/follower) +- [x] Renders index version info in cloud mode +- [x] Follower screen shows the leader's info +- [x] Disable polling, index on the leader, replicate-now transfers the + index, re-enable polling — verified via the replication API + +### Security with BasicAuth — `AdminUiSecurityAuthTest` +- [x] Unauthenticated visit redirects to login; login form authenticates +- [x] Security screen shows authn/authz plugins, users, roles, permissions +- [x] Add a user through the dialog (verified via `/admin/authentication`) +- [x] Add a role for the user through the dialog (verified via API) +- [x] Grant a predefined permission to the role (verified via API) + +### Schema Designer — `AdminUiSchemaDesignerTest` (`@AwaitsFix`) +- [x] Create a new schema, paste a sample doc, analyze; derived field shown — + but the designer backend is too flaky under automation (see Possible UI + bugs), so the test awaits a fix before running by default + +## Deliberately skipped (effort vs value) + +- **JWT/OAuth login flows**: require an external identity provider or heavy + mocking; BasicAuth covers the UI's login/session mechanics. +- **Keystroke-level entry in the security dialogs**: native clicks/keystrokes + into the absolutely-positioned dialogs proved unreliable in headless Chrome; + the dialogs are driven via the Angular controller scope instead. Keyboard + entry is covered by the login form and the other screens' forms. + +## Possible UI bugs to investigate + +Issues surfaced by these tests that look like real bugs, weaknesses or +flakiness in the Admin UI (or its backing APIs) rather than bad test code. +Tests work around them as noted; each deserves investigation and possibly a +JIRA: + +1. **Menu TypeError on per-collection pages**: navigating to any + per-collection screen intermittently logs + `TypeError: Cannot read properties of null (reading 'name')` from + `$scope.showCore` in `js/angular/app.js` — the core selector fires its + change handler with a null core while the menu resolves. Workaround: the + console-error assertion filters this signature. +2. **Collections screen dies without the js-client bundle**: the + `CollectionsV2` service factory (`services.js`) references the `solrApi` + global at injection time; if `libs/solr/index.js` fails to load, the whole + `CollectionsController` fails and the screen is blank. Only + `reloadCollection` is used from that bundle — a lazy/optional lookup would + degrade gracefully. Workaround: tests serve a stub bundle. +3. **Security screen dialogs unreliable under automation**: native clicks on + the Add User toggle and keystrokes into the absolutely-positioned dialog + (jQuery-positioned, `escape-pressed` directive) are dropped in headless + Chrome even though the same interactions work on other screens. May + indicate a focus/z-index issue. Workaround: the test drives the dialog via + the Angular controller scope. +4. **Schema Designer races itself**: creating a schema and analyzing sample + docs transiently fails with `Failed to persist managed schema ... version + mismatch, retry` from its own `prep`/`analyze` calls, surfacing an error + dialog the user has to dismiss. Workaround: the test retries via the + offered Reload Schema button and ignores the designer's own 5xx console + errors. +5. **Plugins screen 500s when metrics are disabled**: `/admin/metrics` with + `wt=prometheus` returns HTTP 500 ("No metrics found in response") when + metrics collection is disabled, instead of a clean error; the Plugins + screen just shows nothing while the console logs the 500. Workaround: + tests enable `metricsEnabled`. +6. **Core overview ping widget logs a 503**: with a configset that has no Review Comment: i think i saw a ticket about "add healthcheck to one of our default configsets" maybe because of this! yes, handle missing item. ########## dev-docs/admin-ui-tests.md: ########## @@ -0,0 +1,226 @@ +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +# Admin UI (AngularJS) Browser Test Plan + +This document tracks browser-based test coverage of the old AngularJS Admin UI +(`solr/webapp/web/`), driven by Selenium WebDriver with headless Chrome. + +## How the tests work + +- Tests live in `solr/webapp/src/test/org/apache/solr/webapp/` and extend + `AdminUiTestBase`, which starts a 2-node `MiniSolrCloudCluster` whose Jetty + nodes also serve the Admin UI (opt-in `JettyConfig.Builder#enableAdminUi`), + then starts a headless Chrome via Selenium WebDriver. +- A locally installed Chrome/Chromium is required; without one, tests skip via + `Assume`. Override discovery with `-Dtests.ui.chrome.binary=/path/to/chrome`. + The matching chromedriver is provisioned (and cached) by Selenium Manager. +- Display assertions compare UI text against live JSON from the same node's + admin APIs — never hardcoded values. +- Tests are grouped per screen/feature, so each screen's display and write + tests live in the same class. +- Most tests run against a 2-node cloud cluster; `AdminUiStandaloneTestBase` + additionally supports standalone (user-managed, no ZooKeeper) nodes, whose + UI differs (no Cloud/Collections/Schema Designer menus; the per-core menu + offers query/replication etc. directly). +- On failure, a screenshot, the page source and the browser console log are + saved into the test temp dir. +- Run with: `./gradlew :solr:webapp:test` + +## Coverage by screen + +### Smoke navigation — `AdminUiSmokeTest` +- [x] Every node-level route (`/`, `~logging`, `~logging/level`, + `~cloud?view=nodes|tree|zkstatus|graph`, `~cores`, `~collections`, + `~schema-designer`, `~security`, `~java-properties`, `~threads`, `login`), + every per-collection route and the per-core routes render their main + content element without severe browser console errors. + +### Dashboard — `AdminUiDashboardTest` +- [x] Versions, JVM info and memory bars vs `/admin/info/system`; security + warning when security is disabled. + +### Node-level screens — `AdminUiNodeScreensTest` +- [x] Java Properties: `java.version` value matches `/admin/info/properties` +- [x] Thread Dump: thread list non-empty, Jetty worker thread shown +- [x] Cloud > Nodes: one row per live node, ports match the cluster +- [x] Cloud > Tree: `live_nodes` and `collections` znodes shown +- [x] Cloud > Graph: d3 SVG renders circles for collection/shard/replicas +- [x] Cloud > ZK Status: status green, ensemble size shown +- [x] Security screen: "not enabled" warning without auth +- [x] Login screen: authentication info page without auth + +### Collections screen — `AdminUiCollectionsScreenTest` +- [x] Collection detail display (shards) +- [x] Create + delete collection via the dialogs (verified via API) +- [x] Create + delete alias via the dialogs (verified via LISTALIASES) +- [x] Add + delete replica via the shard detail (verified via cluster state) +- [x] Reload collection (verified via core start time reset) + +### Query screen — `AdminUiQueryScreenTest` +- [x] `*:*` and `id:` queries via the form, `numFound` in the response +- [x] `rows` and `fl` parameters affect the returned documents +- [x] Paramsets dropdown applies a paramset created via the API +- [x] defType dismax/edismax toggles reveal their parameter fields; edismax + query with `qf` returns the expected results +- [x] Raw query parameters (e.g. an extra `fq`) are applied + +### Documents screen — `AdminUiDocumentsScreenTest` +- [x] Form renders with JSON/XML/CSV document types +- [x] Index a JSON document via the form; becomes searchable + +### Schema screen — `AdminUiSchemaScreenTest` +- [x] Field list browsable, editable-schema action buttons shown +- [x] Field detail shows flags matching `/schema/fields`; term info loads +- [x] Add + delete a field via the dialogs (verified via `/schema/fields`) + +### Paramsets screen — `AdminUiParamsetsScreenTest` +- [x] Form renders +- [x] Create + delete a paramset via the form (verified via `/config/params`) + +### Logging screens — `AdminUiLoggingScreenTest` +- [x] Logger level tree renders +- [x] Set + unset a logger level via the tree (verified via API) +- [x] Events viewer shows a WARN event logged in the server JVM (skips itself + when the node's log watcher is blind due to shared-JVM log4j state, see + Known limitations) + +### Core Admin screen — `AdminUiCoreAdminScreenTest` (cloud) and +### `AdminUiCoreAdminStandaloneTest` (standalone) +- [x] Hosted core listed, matching `/admin/cores` +- [x] Reload core via the button (success indicator) +- [x] Standalone-mode menu differences (no cloud menus; core menu offers + query/replication) +- [x] Add core via the dialog (pre-created instance dir) +- [x] Rename core via the dialog +- [x] Swap cores via the dialog (verified by the indexes exchanging) +- [x] Unload core, accepting the native confirm dialog + +### Per-collection display screens — `AdminUiCollectionScreensTest` +- [x] Analysis: `text_general` tokenizes and lowercases entered text +- [x] Files: tree lists `solrconfig.xml`, file content renders +- [x] Segments: at least one segment rendered after commit +- [x] Plugins/Stats: searcher stats present (needs `metricsEnabled=true`) +- [x] Collection overview: shard info; Core overview: numDocs + +### Stream screen — `AdminUiStreamScreenTest` +- [x] A `search(...)` streaming expression executes and renders all docs + +### SQL screen — `AdminUiSqlScreenTest` +- [x] A SQL query executes through the form and the result grid lists the + documents (the sql module is a test-only dependency of `solr:webapp`) + +### Replication screen — `AdminUiReplicationScreenTest` (cloud) and +### `AdminUiReplicationStandaloneTest` (standalone leader/follower) +- [x] Renders index version info in cloud mode +- [x] Follower screen shows the leader's info +- [x] Disable polling, index on the leader, replicate-now transfers the + index, re-enable polling — verified via the replication API + +### Security with BasicAuth — `AdminUiSecurityAuthTest` +- [x] Unauthenticated visit redirects to login; login form authenticates +- [x] Security screen shows authn/authz plugins, users, roles, permissions +- [x] Add a user through the dialog (verified via `/admin/authentication`) +- [x] Add a role for the user through the dialog (verified via API) +- [x] Grant a predefined permission to the role (verified via API) + +### Schema Designer — `AdminUiSchemaDesignerTest` (`@AwaitsFix`) +- [x] Create a new schema, paste a sample doc, analyze; derived field shown — + but the designer backend is too flaky under automation (see Possible UI + bugs), so the test awaits a fix before running by default + +## Deliberately skipped (effort vs value) + +- **JWT/OAuth login flows**: require an external identity provider or heavy + mocking; BasicAuth covers the UI's login/session mechanics. +- **Keystroke-level entry in the security dialogs**: native clicks/keystrokes + into the absolutely-positioned dialogs proved unreliable in headless Chrome; + the dialogs are driven via the Angular controller scope instead. Keyboard + entry is covered by the login form and the other screens' forms. + +## Possible UI bugs to investigate Review Comment: and I guess we could have a JIRA for all of these seperate? ########## dev-docs/admin-ui-tests.md: ########## @@ -0,0 +1,226 @@ +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +# Admin UI (AngularJS) Browser Test Plan + +This document tracks browser-based test coverage of the old AngularJS Admin UI +(`solr/webapp/web/`), driven by Selenium WebDriver with headless Chrome. + +## How the tests work + +- Tests live in `solr/webapp/src/test/org/apache/solr/webapp/` and extend + `AdminUiTestBase`, which starts a 2-node `MiniSolrCloudCluster` whose Jetty + nodes also serve the Admin UI (opt-in `JettyConfig.Builder#enableAdminUi`), + then starts a headless Chrome via Selenium WebDriver. +- A locally installed Chrome/Chromium is required; without one, tests skip via + `Assume`. Override discovery with `-Dtests.ui.chrome.binary=/path/to/chrome`. + The matching chromedriver is provisioned (and cached) by Selenium Manager. +- Display assertions compare UI text against live JSON from the same node's + admin APIs — never hardcoded values. +- Tests are grouped per screen/feature, so each screen's display and write + tests live in the same class. +- Most tests run against a 2-node cloud cluster; `AdminUiStandaloneTestBase` + additionally supports standalone (user-managed, no ZooKeeper) nodes, whose + UI differs (no Cloud/Collections/Schema Designer menus; the per-core menu + offers query/replication etc. directly). +- On failure, a screenshot, the page source and the browser console log are + saved into the test temp dir. +- Run with: `./gradlew :solr:webapp:test` + +## Coverage by screen + +### Smoke navigation — `AdminUiSmokeTest` +- [x] Every node-level route (`/`, `~logging`, `~logging/level`, + `~cloud?view=nodes|tree|zkstatus|graph`, `~cores`, `~collections`, + `~schema-designer`, `~security`, `~java-properties`, `~threads`, `login`), + every per-collection route and the per-core routes render their main + content element without severe browser console errors. + +### Dashboard — `AdminUiDashboardTest` +- [x] Versions, JVM info and memory bars vs `/admin/info/system`; security + warning when security is disabled. + +### Node-level screens — `AdminUiNodeScreensTest` +- [x] Java Properties: `java.version` value matches `/admin/info/properties` +- [x] Thread Dump: thread list non-empty, Jetty worker thread shown +- [x] Cloud > Nodes: one row per live node, ports match the cluster +- [x] Cloud > Tree: `live_nodes` and `collections` znodes shown +- [x] Cloud > Graph: d3 SVG renders circles for collection/shard/replicas +- [x] Cloud > ZK Status: status green, ensemble size shown +- [x] Security screen: "not enabled" warning without auth +- [x] Login screen: authentication info page without auth + +### Collections screen — `AdminUiCollectionsScreenTest` +- [x] Collection detail display (shards) +- [x] Create + delete collection via the dialogs (verified via API) +- [x] Create + delete alias via the dialogs (verified via LISTALIASES) +- [x] Add + delete replica via the shard detail (verified via cluster state) +- [x] Reload collection (verified via core start time reset) + +### Query screen — `AdminUiQueryScreenTest` +- [x] `*:*` and `id:` queries via the form, `numFound` in the response +- [x] `rows` and `fl` parameters affect the returned documents +- [x] Paramsets dropdown applies a paramset created via the API +- [x] defType dismax/edismax toggles reveal their parameter fields; edismax + query with `qf` returns the expected results +- [x] Raw query parameters (e.g. an extra `fq`) are applied + +### Documents screen — `AdminUiDocumentsScreenTest` +- [x] Form renders with JSON/XML/CSV document types +- [x] Index a JSON document via the form; becomes searchable + +### Schema screen — `AdminUiSchemaScreenTest` +- [x] Field list browsable, editable-schema action buttons shown +- [x] Field detail shows flags matching `/schema/fields`; term info loads +- [x] Add + delete a field via the dialogs (verified via `/schema/fields`) + +### Paramsets screen — `AdminUiParamsetsScreenTest` +- [x] Form renders +- [x] Create + delete a paramset via the form (verified via `/config/params`) + +### Logging screens — `AdminUiLoggingScreenTest` +- [x] Logger level tree renders +- [x] Set + unset a logger level via the tree (verified via API) +- [x] Events viewer shows a WARN event logged in the server JVM (skips itself + when the node's log watcher is blind due to shared-JVM log4j state, see + Known limitations) + +### Core Admin screen — `AdminUiCoreAdminScreenTest` (cloud) and +### `AdminUiCoreAdminStandaloneTest` (standalone) +- [x] Hosted core listed, matching `/admin/cores` +- [x] Reload core via the button (success indicator) +- [x] Standalone-mode menu differences (no cloud menus; core menu offers + query/replication) +- [x] Add core via the dialog (pre-created instance dir) +- [x] Rename core via the dialog +- [x] Swap cores via the dialog (verified by the indexes exchanging) +- [x] Unload core, accepting the native confirm dialog + +### Per-collection display screens — `AdminUiCollectionScreensTest` +- [x] Analysis: `text_general` tokenizes and lowercases entered text +- [x] Files: tree lists `solrconfig.xml`, file content renders +- [x] Segments: at least one segment rendered after commit +- [x] Plugins/Stats: searcher stats present (needs `metricsEnabled=true`) +- [x] Collection overview: shard info; Core overview: numDocs + +### Stream screen — `AdminUiStreamScreenTest` +- [x] A `search(...)` streaming expression executes and renders all docs + +### SQL screen — `AdminUiSqlScreenTest` +- [x] A SQL query executes through the form and the result grid lists the + documents (the sql module is a test-only dependency of `solr:webapp`) + +### Replication screen — `AdminUiReplicationScreenTest` (cloud) and +### `AdminUiReplicationStandaloneTest` (standalone leader/follower) +- [x] Renders index version info in cloud mode +- [x] Follower screen shows the leader's info +- [x] Disable polling, index on the leader, replicate-now transfers the + index, re-enable polling — verified via the replication API + +### Security with BasicAuth — `AdminUiSecurityAuthTest` +- [x] Unauthenticated visit redirects to login; login form authenticates +- [x] Security screen shows authn/authz plugins, users, roles, permissions +- [x] Add a user through the dialog (verified via `/admin/authentication`) +- [x] Add a role for the user through the dialog (verified via API) +- [x] Grant a predefined permission to the role (verified via API) + +### Schema Designer — `AdminUiSchemaDesignerTest` (`@AwaitsFix`) +- [x] Create a new schema, paste a sample doc, analyze; derived field shown — + but the designer backend is too flaky under automation (see Possible UI + bugs), so the test awaits a fix before running by default + +## Deliberately skipped (effort vs value) + +- **JWT/OAuth login flows**: require an external identity provider or heavy + mocking; BasicAuth covers the UI's login/session mechanics. +- **Keystroke-level entry in the security dialogs**: native clicks/keystrokes + into the absolutely-positioned dialogs proved unreliable in headless Chrome; + the dialogs are driven via the Angular controller scope instead. Keyboard + entry is covered by the login form and the other screens' forms. + +## Possible UI bugs to investigate + +Issues surfaced by these tests that look like real bugs, weaknesses or +flakiness in the Admin UI (or its backing APIs) rather than bad test code. +Tests work around them as noted; each deserves investigation and possibly a +JIRA: + +1. **Menu TypeError on per-collection pages**: navigating to any + per-collection screen intermittently logs + `TypeError: Cannot read properties of null (reading 'name')` from + `$scope.showCore` in `js/angular/app.js` — the core selector fires its + change handler with a null core while the menu resolves. Workaround: the + console-error assertion filters this signature. +2. **Collections screen dies without the js-client bundle**: the + `CollectionsV2` service factory (`services.js`) references the `solrApi` + global at injection time; if `libs/solr/index.js` fails to load, the whole + `CollectionsController` fails and the screen is blank. Only + `reloadCollection` is used from that bundle — a lazy/optional lookup would + degrade gracefully. Workaround: tests serve a stub bundle. +3. **Security screen dialogs unreliable under automation**: native clicks on + the Add User toggle and keystrokes into the absolutely-positioned dialog + (jQuery-positioned, `escape-pressed` directive) are dropped in headless + Chrome even though the same interactions work on other screens. May + indicate a focus/z-index issue. Workaround: the test drives the dialog via + the Angular controller scope. +4. **Schema Designer races itself**: creating a schema and analyzing sample + docs transiently fails with `Failed to persist managed schema ... version + mismatch, retry` from its own `prep`/`analyze` calls, surfacing an error + dialog the user has to dismiss. Workaround: the test retries via the + offered Reload Schema button and ignores the designer's own 5xx console + errors. +5. **Plugins screen 500s when metrics are disabled**: `/admin/metrics` with + `wt=prometheus` returns HTTP 500 ("No metrics found in response") when + metrics collection is disabled, instead of a clean error; the Plugins + screen just shows nothing while the console logs the 500. Workaround: + tests enable `metricsEnabled`. +6. **Core overview ping widget logs a 503**: with a configset that has no + healthcheck file, the ping status call answers 503 and the console shows a + resource-load error on every visit; the widget could handle "healthcheck + not configured" gracefully. Workaround: allowed in the affected tests. +7. **Reload success indicator is a 1-second flash**: the Collections screen's + reload button only flags success via a CSS class for one second, which is + easy to miss (and impossible to assert on reliably). Workaround: the test + verifies the reload via the core start time instead. +8. **ui-grid icon font is missing from the webapp**: `css/angular/ui-grid.min.css` + references `fonts/ui-grid.woff` (and .ttf/.eot), but no such font files are + shipped anywhere under `solr/webapp/web` — the SQL screen's result grid + logs a 404 for it in production too, and grid icons render as boxes. + Workaround: the console-error assertion filters this 404. + +## Known limitations Review Comment: some o fthese items *might* be resolved in the "migrate to v2" work, jthere were some weaknesses in events identified... ########## solr/webapp/src/test/org/apache/solr/webapp/AdminUiTestBase.java: ########## @@ -0,0 +1,534 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.webapp; + +import com.carrotsearch.randomizedtesting.ThreadFilter; +import com.carrotsearch.randomizedtesting.annotations.ThreadLeakFilters; +import com.carrotsearch.randomizedtesting.annotations.ThreadLeakLingering; +import jakarta.servlet.http.HttpServlet; +import jakarta.servlet.http.HttpServletRequest; +import jakarta.servlet.http.HttpServletResponse; +import java.io.File; +import java.io.IOException; +import java.lang.invoke.MethodHandles; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.time.Duration; +import java.util.Arrays; +import java.util.List; +import java.util.function.BooleanSupplier; +import java.util.function.Function; +import java.util.logging.Level; +import org.apache.lucene.tests.util.QuickPatchThreadsFilter; +import org.apache.lucene.util.SuppressForbidden; +import org.apache.solr.SolrIgnoredThreadsFilter; +import org.apache.solr.SolrTestCaseJ4; +import org.apache.solr.client.solrj.SolrClient; +import org.apache.solr.client.solrj.SolrRequest; +import org.apache.solr.client.solrj.SolrServerException; +import org.apache.solr.client.solrj.request.CollectionAdminRequest; +import org.apache.solr.client.solrj.request.GenericSolrRequest; +import org.apache.solr.cloud.SolrCloudTestCase; +import org.apache.solr.common.params.SolrParams; +import org.apache.solr.common.util.EnvUtils; +import org.apache.solr.common.util.NamedList; +import org.apache.solr.embedded.JettyConfig; +import org.apache.solr.embedded.JettySolrRunner; +import org.apache.solr.util.ExternalPaths; +import org.eclipse.jetty.ee10.servlet.ServletHolder; +import org.junit.AfterClass; +import org.junit.Assume; +import org.junit.BeforeClass; +import org.junit.Rule; +import org.junit.rules.TestRule; +import org.junit.rules.TestWatcher; +import org.junit.runner.Description; +import org.openqa.selenium.By; +import org.openqa.selenium.NoSuchElementException; +import org.openqa.selenium.OutputType; +import org.openqa.selenium.StaleElementReferenceException; +import org.openqa.selenium.TakesScreenshot; +import org.openqa.selenium.WebDriver; +import org.openqa.selenium.WebDriverException; +import org.openqa.selenium.WebElement; +import org.openqa.selenium.chrome.ChromeDriver; +import org.openqa.selenium.chrome.ChromeOptions; +import org.openqa.selenium.logging.LogEntry; +import org.openqa.selenium.logging.LogType; +import org.openqa.selenium.logging.LoggingPreferences; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Base class for browser-based tests of the AngularJS Admin UI. + * + * <p>Starts a {@link SolrCloudTestCase} mini-cluster whose Jetty nodes also serve the Admin UI + * static files (see {@code JettyConfig.Builder#enableAdminUi(boolean)}), then drives the UI with a + * headless Chrome via Selenium WebDriver. + * + * <p>The tests require a locally installed Chrome/Chromium browser. Discovery order: the {@code + * tests.ui.chrome.binary} system property, the {@code CHROME_BIN} environment variable, then a list + * of well-known install locations. When no browser is found, all tests in the class are skipped via + * {@link Assume}. The matching chromedriver is provisioned by Selenium Manager, which may download + * it on first use (cached under {@code ~/.cache/selenium}); if that fails (e.g. offline), tests are + * likewise skipped. + */ [email protected](bugUrl = "Admin UI browser tests drive plain http") +@ThreadLeakFilters( + defaultFilters = true, + filters = { + SolrIgnoredThreadsFilter.class, + QuickPatchThreadsFilter.class, + AdminUiTestBase.WebDriverThreadsFilter.class + }) +@ThreadLeakLingering(linger = 5000) +public abstract class AdminUiTestBase extends SolrCloudTestCase { + + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + + protected static final Duration WAIT_TIMEOUT = Duration.ofSeconds(15); + + protected static WebDriver driver; + + /** Base url of the first node, e.g. {@code http://127.0.0.1:PORT/solr} */ + protected static String baseUrl; + + /** + * Optional security.json for the cluster. Subclasses assign this in their {@code @BeforeClass} + * (which runs after this class's browser-starting one, but before the cluster starts lazily on + * first use). Never assign it in a {@code static} block: test runners may load all test classes + * up front, so static initializers of one class can run long before its suite executes. + */ + protected static String securityJson; + + /** + * When true (set by {@code AdminUiStandaloneTestBase}), no cloud cluster is started; the test + * class starts its own standalone {@link JettySolrRunner}(s), assigns {@link #standaloneJetty} + * and {@link #baseUrl}, and stops them again. + */ + protected static boolean standaloneMode = false; + + /** The standalone node backing {@link #adminApi} when {@link #standaloneMode} is set. */ + protected static JettySolrRunner standaloneJetty; + + /** + * Serves a minimal stand-in for the generated js-client bundle ({@code libs/solr/index.js}), + * which only exists inside the built webapp, not in the source tree tests serve from. The + * AngularJS {@code CollectionsV2} service fails to instantiate without the {@code solrApi} + * global, taking the whole Collections screen down with it. Only the small API surface the + * AngularJS UI actually uses is stubbed. + */ + public static class StubJsClientServlet extends HttpServlet { + @Override + protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException { + resp.setContentType("text/javascript"); + resp.getOutputStream() + .write( + ("var solrApi = {\n" + + " ApiClient: { instance: { basePath: '/api', defaultHeaders: {} } },\n" + + " CollectionsApi: function() {\n" + + " this.reloadCollection = function(name, callback) {\n" + + " var xhr = new XMLHttpRequest();\n" + + " xhr.open('POST', '/api/collections/' + name + '/reload');\n" + + " xhr.setRequestHeader('Content-Type', 'application/json');\n" + + " xhr.onload = function() { callback(null, null, {status: xhr.status}); };\n" + + " xhr.onerror = function() { callback(new Error('reload failed'), null, {status: xhr.status}); };\n" + + " xhr.send('{}');\n" + + " };\n" + + " }\n" + + "};\n") + .getBytes(StandardCharsets.UTF_8)); + } + } + + /** Ignores threads spawned by Selenium and the JDK http client it uses. */ + public static class WebDriverThreadsFilter implements ThreadFilter { + @Override + public boolean reject(Thread t) { + String name = t.getName(); + // JDK java.net.http client worker/selector threads (used by Selenium) are daemon + // threads in a shared pool that outlive WebDriver.quit() + return name.startsWith("HttpClient-") + // reaps the external chromedriver/chrome processes + || name.startsWith("process reaper") + // selenium driver-service startup checker pool, terminates on its own + || name.startsWith("UrlChecker-") + // selenium's chromedriver stdout/stderr pump, stops when the process exits + || name.startsWith("External Process Output Forwarder") + // JDK-internal scheduler backing CompletableFuture timeouts, lives forever + || name.equals("CompletableFutureDelayScheduler"); + } + } + + @BeforeClass + @SuppressForbidden(reason = "Selenium's logging preferences API uses java.util.logging levels") + public static void startClusterAndBrowser() throws Exception { + Path chrome = findChromeBinary(); + Assume.assumeTrue( + "No Chrome/Chromium binary found (set -Dtests.ui.chrome.binary=...), skipping UI tests", + chrome != null); + + // metrics are off by default in test clusters, but UI screens (e.g. Plugins) need them; + // restored after the class by SolrTestCase's SystemPropertiesRestoreRule + System.setProperty("metricsEnabled", "true"); + // the cluster starts lazily via ensureCloudCluster(), after subclass @BeforeClass + // methods have had the chance to configure securityJson or standalone mode + + ChromeOptions options = new ChromeOptions(); + options.setBinary(chrome.toString()); + options.addArguments( + "--headless=new", + "--window-size=1440,1024", + "--disable-gpu", + "--no-sandbox", + "--disable-dev-shm-usage"); + LoggingPreferences logPrefs = new LoggingPreferences(); + logPrefs.enable(LogType.BROWSER, Level.ALL); + options.setCapability("goog:loggingPrefs", logPrefs); + try { + driver = new ChromeDriver(options); + } catch (WebDriverException e) { + Assume.assumeNoException( + "Could not start ChromeDriver (chromedriver missing and not downloadable?)", e); + } + driver.manage().timeouts().pageLoadTimeout(Duration.ofSeconds(30)); + } + + /** Starts the 2-node cloud cluster serving the UI, unless already started. */ + protected static void ensureCloudCluster() { + if (standaloneMode || cluster != null) { + return; + } + try { + var clusterBuilder = + configureCluster(2).withJettyConfig(AdminUiTestBase::configureJettyForUi); + if (securityJson != null) { + clusterBuilder.withSecurityJson(securityJson); + } + clusterBuilder.configure(); + baseUrl = cluster.getJettySolrRunner(0).getBaseUrl().toString(); + } catch (Exception e) { + throw new RuntimeException("Could not start UI test cluster", e); + } + } + + /** Configures a Jetty node to serve the Admin UI plus the js-client stub. */ + protected static void configureJettyForUi(JettyConfig.Builder jetty) { + jetty + .enableAdminUi(true) + // exact-path mapping takes precedence over the static /libs/* servlet + .withServlet(new ServletHolder(new StubJsClientServlet()), "/libs/solr/index.js"); + } + + @AfterClass + public static void stopBrowser() { + // reset the static per-class configuration: several test classes run in the same + // JVM, and flags set by one class's static initializer must not leak into the next + standaloneMode = false; + standaloneJetty = null; + securityJson = null; + if (driver != null) { + try { + driver.quit(); + } finally { + driver = null; + baseUrl = null; + } + } + } + + /** Captures a screenshot and the page source when a test fails, for post-mortem debugging. */ + @Rule + public final TestRule screenshotOnFailure = + new TestWatcher() { + @Override + protected void failed(Throwable e, Description description) { + if (driver == null) return; + try { + Path dir = createTempDir("ui-failure-" + description.getMethodName()); + byte[] png = ((TakesScreenshot) driver).getScreenshotAs(OutputType.BYTES); + Files.write(dir.resolve("screenshot.png"), png); + Files.writeString(dir.resolve("page.html"), driver.getPageSource()); + StringBuilder console = new StringBuilder(); + for (LogEntry entry : driver.manage().logs().get(LogType.BROWSER).getAll()) { + console.append(entry.getLevel()).append(' ').append(entry.getMessage()).append('\n'); + } + Files.writeString(dir.resolve("console.log"), console.toString()); + log.error("UI test failure artifacts saved to {}", dir); + } catch (Exception suppressed) { + log.warn("Could not save UI failure artifacts", suppressed); + } + } + }; + + /** + * Navigates to an Admin UI page and waits for a screen-specific anchor element to be visible. + * + * @param route the Angular hash route without leading {@code #/}, e.g. {@code ""} (dashboard), + * {@code "~cloud"} or {@code "collection1/query"} + * @param anchor a locator for an element that indicates the screen has rendered + * @return the anchor element + */ + protected static WebElement openPage(String route, By anchor) { + ensureCloudCluster(); + driver.get(baseUrl + "/index.html#/" + route); + return waitFor(anchor); + } + + /** Waits for the given element to be visible, up to {@link #WAIT_TIMEOUT}. */ + protected static WebElement waitFor(By locator) { + return poll(locator, el -> el.isDisplayed() ? el : null, "visible element"); + } + + /** Waits until the given element has non-blank text, and returns the text. */ + protected static String waitForText(By locator) { + return poll( + locator, + el -> { + String text = el.getText(); + return el.isDisplayed() && !text.isBlank() ? text : null; + }, + "non-empty text"); + } + + /** + * Polls the given element until {@code condition} returns non-null (a fresh lookup each round, so + * elements replaced by Angular re-renders are tolerated), failing after {@link #WAIT_TIMEOUT}. + */ + private static <T> T poll(By locator, Function<WebElement, T> condition, String description) { + long deadlineNanos = System.nanoTime() + WAIT_TIMEOUT.toNanos(); + WebDriverException lastException = null; + while (System.nanoTime() < deadlineNanos) { + try { + T result = condition.apply(driver.findElement(locator)); + if (result != null) { + return result; + } + lastException = null; + } catch (NoSuchElementException | StaleElementReferenceException e) { + lastException = e; + } + try { + Thread.sleep(200); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + break; + } + } + throw new AssertionError( + "Timed out waiting for " + description + " at " + locator, lastException); + } + + /** + * Issues a GET request to the given admin path (e.g. {@code /admin/info/system}) on the same node + * the browser talks to, and returns the parsed response. Used to fetch the expected values that + * the UI should display. + */ + protected static NamedList<Object> adminApi(String path, SolrParams params) + throws IOException, SolrServerException { + ensureCloudCluster(); + JettySolrRunner jetty = standaloneMode ? standaloneJetty : cluster.getJettySolrRunner(0); + try (SolrClient client = jetty.newClient()) { + return client.request(new GenericSolrRequest(SolrRequest.METHOD.GET, path, params)); + } + } + + /** + * Uploads the default configset under the collection's name and creates the collection. A + * single-replica collection is pinned to the node the browser talks to, so core-level screens + * find its core locally. + */ + protected static void createFixtureCollection(String name, int numShards, int numReplicas) + throws Exception { + ensureCloudCluster(); + cluster.uploadConfigSet(ExternalPaths.DEFAULT_CONFIGSET, name); + CollectionAdminRequest.Create create = Review Comment: interesting, we have a SolrJ way of doing this, but I think we also generate a Java stub called `CreateCollection` that could be used here. ########## dev-docs/admin-ui-tests.md: ########## @@ -0,0 +1,177 @@ +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +# Admin UI (AngularJS) Browser Test Plan + +This document tracks browser-based test coverage of the old AngularJS Admin UI +(`solr/webapp/web/`), driven by Selenium WebDriver with headless Chrome. + +## How the tests work + +- Tests live in `solr/webapp/src/test/org/apache/solr/webapp/` and extend + `AdminUiTestBase`, which starts a 2-node `MiniSolrCloudCluster` whose Jetty + nodes also serve the Admin UI (opt-in `JettyConfig.Builder#enableAdminUi`), + then starts a headless Chrome via Selenium WebDriver. +- A locally installed Chrome/Chromium is required; without one, tests skip via + `Assume`. Override discovery with `-Dtests.ui.chrome.binary=/path/to/chrome`. + The matching chromedriver is provisioned (and cached) by Selenium Manager. +- Display assertions compare UI text against live JSON from the same node's + admin APIs — never hardcoded values. +- Tests are grouped per screen/feature, so each screen's display and write + tests live in the same class. +- Most tests run against a 2-node cloud cluster; `AdminUiStandaloneTestBase` + additionally supports standalone (user-managed, no ZooKeeper) nodes, whose + UI differs (no Cloud/Collections/Schema Designer menus; the per-core menu + offers query/replication etc. directly). +- On failure, a screenshot, the page source and the browser console log are + saved into the test temp dir. +- Run with: `./gradlew :solr:webapp:test` + +## Coverage by screen + +### Smoke navigation — `AdminUiSmokeTest` +- [x] Every node-level route (`/`, `~logging`, `~logging/level`, + `~cloud?view=nodes|tree|zkstatus|graph`, `~cores`, `~collections`, + `~schema-designer`, `~security`, `~java-properties`, `~threads`, `login`), + every per-collection route and the per-core routes render their main + content element without severe browser console errors. + +### Dashboard — `AdminUiDashboardTest` +- [x] Versions, JVM info and memory bars vs `/admin/info/system`; security + warning when security is disabled. + +### Node-level screens — `AdminUiNodeScreensTest` +- [x] Java Properties: `java.version` value matches `/admin/info/properties` +- [x] Thread Dump: thread list non-empty, Jetty worker thread shown +- [x] Cloud > Nodes: one row per live node, ports match the cluster +- [x] Cloud > Tree: `live_nodes` and `collections` znodes shown +- [x] Cloud > Graph: d3 SVG renders circles for collection/shard/replicas +- [x] Cloud > ZK Status: status green, ensemble size shown +- [x] Security screen: "not enabled" warning without auth +- [x] Login screen: authentication info page without auth + +### Collections screen — `AdminUiCollectionsScreenTest` +- [x] Collection detail display (shards) +- [x] Create + delete collection via the dialogs (verified via API) +- [x] Create + delete alias via the dialogs (verified via LISTALIASES) +- [x] Add + delete replica via the shard detail (verified via cluster state) +- [x] Reload collection (verified via core start time reset) + +### Query screen — `AdminUiQueryScreenTest` +- [x] `*:*` and `id:` queries via the form, `numFound` in the response +- [x] `rows` and `fl` parameters affect the returned documents +- [x] Paramsets dropdown applies a paramset created via the API +- [x] defType dismax/edismax toggles reveal their parameter fields; edismax + query with `qf` returns the expected results +- [x] Raw query parameters (e.g. an extra `fq`) are applied + +### Documents screen — `AdminUiDocumentsScreenTest` +- [x] Form renders with JSON/XML/CSV document types +- [x] Index a JSON document via the form; becomes searchable + +### Schema screen — `AdminUiSchemaScreenTest` +- [x] Field list browsable, editable-schema action buttons shown +- [x] Field detail shows flags matching `/schema/fields`; term info loads +- [x] Add + delete a field via the dialogs (verified via `/schema/fields`) + +### Paramsets screen — `AdminUiParamsetsScreenTest` +- [x] Form renders +- [x] Create + delete a paramset via the form (verified via `/config/params`) + +### Logging screens — `AdminUiLoggingScreenTest` +- [x] Logger level tree renders +- [x] Set + unset a logger level via the tree (verified via API) +- [x] Events viewer shows a WARN event logged in the server JVM (skips itself + when the node's log watcher is blind due to shared-JVM log4j state, see + Known limitations) + +### Core Admin screen — `AdminUiCoreAdminScreenTest` (cloud) and +### `AdminUiCoreAdminStandaloneTest` (standalone) +- [x] Hosted core listed, matching `/admin/cores` +- [x] Reload core via the button (success indicator) +- [x] Standalone-mode menu differences (no cloud menus; core menu offers + query/replication) +- [x] Add core via the dialog (pre-created instance dir) +- [x] Rename core via the dialog +- [x] Swap cores via the dialog (verified by the indexes exchanging) +- [x] Unload core, accepting the native confirm dialog + +### Per-collection display screens — `AdminUiCollectionScreensTest` +- [x] Analysis: `text_general` tokenizes and lowercases entered text +- [x] Files: tree lists `solrconfig.xml`, file content renders +- [x] Segments: at least one segment rendered after commit +- [x] Plugins/Stats: searcher stats present (needs `metricsEnabled=true`) +- [x] Collection overview: shard info; Core overview: numDocs + +### Stream screen — `AdminUiStreamScreenTest` +- [x] A `search(...)` streaming expression executes and renders all docs + +### SQL screen — `AdminUiSqlScreenTest` +- [x] A SQL query executes through the form and the result grid lists the + documents (the sql module is a test-only dependency of `solr:webapp`) + +### Replication screen — `AdminUiReplicationScreenTest` (cloud) and +### `AdminUiReplicationStandaloneTest` (standalone leader/follower) +- [x] Renders index version info in cloud mode +- [x] Follower screen shows the leader's info +- [x] Disable polling, index on the leader, replicate-now transfers the + index, re-enable polling — verified via the replication API + +### Security with BasicAuth — `AdminUiSecurityAuthTest` +- [x] Unauthenticated visit redirects to login; login form authenticates +- [x] Security screen shows authn/authz plugins, users, roles, permissions +- [x] Add a user through the dialog (verified via `/admin/authentication`) +- [x] Add a role for the user through the dialog (verified via API) +- [x] Grant a predefined permission to the role (verified via API) + +### Schema Designer — `AdminUiSchemaDesignerTest` (`@AwaitsFix`) +- [x] Create a new schema, paste a sample doc, analyze; derived field shown — + but the designer backend is too flaky under automation + (see [SOLR-18347](https://issues.apache.org/jira/browse/SOLR-18347)), + so the test awaits a fix before running by default + +## Deliberately skipped (effort vs value) + +- **JWT/OAuth login flows**: require an external identity provider or heavy + mocking; BasicAuth covers the UI's login/session mechanics. +- **Keystroke-level entry in the security dialogs**: native clicks/keystrokes + into the absolutely-positioned dialogs proved unreliable in headless Chrome; + the dialogs are driven via the Angular controller scope instead. Keyboard + entry is covered by the login form and the other screens' forms. + +## Known limitations + +- The generated js-client bundle (`libs/solr/index.js`) only exists inside the + built WAR, not in the source tree tests serve from. `AdminUiTestBase` serves + a minimal stub defining the `solrApi` global (only `reloadCollection` is used + by the AngularJS UI); a future improvement could serve the real bundle when + it has been built. +- Every test cluster in the JVM registers a log-watcher appender under the same + name in the shared log4j config, so a later cluster's watcher can be blind; + the events-viewer test detects this via the API and skips itself. +- The shared menu code logs a benign + `TypeError: Cannot read properties of null (reading 'name')` from + `$scope.showCore` while the per-collection menu resolves (filtered in the + console-error assertion; candidate for a JIRA). +- The core overview ping widget answers 503 when the configset has no Review Comment: Should this item have a seperate JIRA to get fixed? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
