samuhasavak5 opened a new pull request, #1413: URL: https://github.com/apache/knox/pull/1413
KNOX-3474 - Add spark4historyui Knox service definition Depends on #1411, #1412 ## What changes were proposed in this pull request? This PR adds upstream Knox support for the dedicated Spark 4 History Server Web UI (role: SPARK4HISTORYUI). Problem: Knox has no service definition for Spark 4 History Server. Deployments running a separate Spark 4 History Server cannot be proxied or discovered through Knox, YARN UI links to Spark 4 history jobs do not route correctly, and the Knox Homepage cannot show a dedicated Spark 4 History tile. Solution: Introduce spark4historyui by mirroring the spark3historyui service definition with Spark 4 naming, CM discovery, admin UI registration, homepage logo, YARN rewrite rules, and unit tests. ### Changes | File | Change | |------|--------| | gateway-service-definitions/.../spark4historyui/3.0.0/service.xml | New service definition (metadata, routes) | | gateway-service-definitions/.../spark4historyui/3.0.0/rewrite.xml | Inbound/outbound rewrite rules for REST API and UI | | gateway-discovery-cm/.../Spark4HistoryUIServiceModelGenerator.java | CM discovery for SPARK4HISTORYUI | | gateway-discovery-cm/.../Spark4HistoryUIServiceModelGeneratorTest.java | Unit tests for model generator | | gateway-discovery-cm/.../ServiceModelGenerator (SPI) | Register Spark4HistoryUIServiceModelGenerator | | ClouderaManagerServiceDiscoveryTest.java | Add testSpark4HistoryUIDiscovery and testSpark4HistoryUIDiscoverySSL | | gateway-service-definitions/.../yarnui/2.7.0/rewrite.xml | Route ?spark4=true history links to /spark4history/ | | gateway-service-definitions/.../yarnuiv2/3.0.0/rewrite.xml | Same YARN rewrite rule | | gateway-admin-ui/.../new-desc-wizard.component.ts | Add SPARK4HISTORYUI to descriptor wizard | | knox-homepage-ui/home/assets/service-logos/spark4historyui.png | Homepage logo for service role SPARK4HISTORYUI | ### Service details | Item | Value | |------|-------| | Service role | SPARK4HISTORYUI | | Service name / path | spark4history | | CM service type | SPARK4_ON_YARN | | CM role type | SPARK4_YARN_HISTORY_SERVER | | Metadata type | UI | | Homepage context | /spark4history/ | ### Design notes * Mirrors spark3historyui with Spark 4 identifiers only * Additive only — no changes to spark3historyui or sparkhistoryui service definitions * YARN rewrite — spark4=true query param routes to spark4history (parallel to CDH spark3=true → spark3history pattern) * Homepage logo — spark4historyui.png (same Spark branding as Spark 3 History) ### Compatibility * No changes to existing sparkhistoryui or spark3historyui behavior * New service is opt-in via topology / CM discovery configuration * Backward compatible for clusters that do not deploy Spark 4 History Server JIRA: https://issues.apache.org/jira/browse/KNOX-3474 ## How was this patch tested? ### Build mvn -s mvn-settings-central.xml -U -pl gateway-service-definitions,gateway-discovery-cm -am test -DskipITs Result: BUILD SUCCESS ### Unit tests | Test | Coverage | |------|----------| | Spark4HistoryUIServiceModelGeneratorTest | CM model generator metadata and URL generation | | ClouderaManagerServiceDiscoveryTest.testSpark4HistoryUIDiscovery | HTTP discovery for SPARK4HISTORYUI | | ClouderaManagerServiceDiscoveryTest.testSpark4HistoryUIDiscoverySSL | HTTPS discovery for SPARK4HISTORYUI | | Full reactor (gateway-service-definitions, gateway-discovery-cm + dependencies) | 351 tests passed, 0 failures | ### Manual verification * Knox Homepage on spark4knoxui cluster shows "Spark 4 History Server Web UI" tile with logo * Tile click opens Spark 4 History UI through Knox gateway path /spark4history/ * YARN UI history links with spark4=true route to /spark4history/history/ ## Integration Tests No new integration test workflow added. Coverage is provided by existing unit tests in gateway-discovery-cm and gateway-service-definitions. ## UI changes Yes — Knox Homepage shows a new Spark 4 History Server tile with logo when SPARK4HISTORYUI is configured in the topology. ## Was this patch authored or co-authored using generative AI tooling? Yes, this was co-authored using Cursor to help port the service definition from spark3historyui, discovery generator, tests, YARN rewrite rules, and PR description. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
