[
https://issues.apache.org/jira/browse/KNOX-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Kumar M updated KNOX-3474:
-------------------------------
Description:
*Summary*
Add upstream Knox support for the Spark 4 History Server Web UI (role:
SPARK4HISTORYUI).
*Motivation*
Knox already supports spark3historyui (SPARK3HISTORYUI). Spark 4 deployments
need an equivalent service definition so the History Server UI is reachable
through Knox and linked correctly from YARN ResourceManager UI.
*Implementation*
Base on existing spark3historyui/3.0.0/ pattern:
| Item | Value |
|------|-------|
| Service role | SPARK4HISTORYUI |
| Service name / URL path | spark4history |
| Metadata type | UI |
| shortDesc | Spark 4 History Server Web UI |
| CM discovery | Spark4HistoryUIServiceModelGenerator (extend
SparkHistoryUIServiceModelGenerator) |
| CM service type | SPARK4_ON_YARN (align with CM Spark 4 service type) |
| CM role type | SPARK4_YARN_HISTORY_SERVER (align with CM Spark 4 role type) |
*Scope*
*1. Service definition*
- gateway-service-definitions/services/spark4historyui/3.0.0/service.xml
- gateway-service-definitions/services/spark4historyui/3.0.0/rewrite.xml
- Inbound/outbound rewrite rules for /spark4history/, /history/, jobs pages,
SSO header filters
*2. Cloudera Manager discovery*
- Spark4HistoryUIServiceModelGenerator.java + unit tests
- META-INF/services ServiceModelGenerator registration
- ClouderaManagerServiceDiscoveryTest (HTTP and SSL port variants)
*3. YARN UI cross-links (required for RM → Spark 4 History navigation)*
- gateway-service-definitions/services/yarnui/2.7.0/rewrite.xml
- Add spark4history redirect rule parallel to existing spark3history rule
- Pattern: *://*:*/history/\{**}?spark4=true?\{**} →
\{$frontend[url]}/spark4history/history/\{**}?\{**}
- gateway-service-definitions/services/yarnuiv2/3.0.0/rewrite.xml
- Same spark4history rule as above
*4. Admin / configuration*
- gateway-admin-ui: add SPARK4HISTORYUI to service role list
- Homepage icon for Spark 4 History Server (if icon metadata support is
included)
*Out of scope*
- Spark History Server SPNEGO / authentication configuration
- Spark 4 parcel installation issues
*Acceptance criteria*
- /spark4history/** routes to Spark 4 History Server backend through Knox
- CM discovery resolves SPARK4HISTORYUI URLs (HTTP and HTTPS)
- YARN RM UI links with ?spark4=true route to Knox spark4history path
- Service tile appears on Knox Homepage
- Unit tests pass
was:
Add upstream Knox support for Spark 4 History Server Web UI (role:
SPARK4HISTORYUI).
Implementation:
- New service definition spark4historyui/ (based on spark3historyui/3.0.0/)
- Service role: SPARK4HISTORYUI, path: /spark4history/
- CM discovery generator for Spark 4 History Server role
- Update YARN UI rewrite rules:
- gateway-service-definitions/.../yarnui/2.7.0/rewrite.xml
- gateway-service-definitions/.../yarnuiv2/3.0.0/rewrite.xml
(add spark4history links, parallel to existing spark3history entries)
- Homepage icon for Spark 4 History Server
Source: CDH/knox PR #796 + logo commit c93158dd
Note: Spark 4 SHS SPNEGO install issue is separate (CDEP workaround:
--no-spark-shs-spnego); this JIRA is Knox routing/UI only.
> Add Knox service definition for Spark 4 History Server UI (spark4historyui)
> ---------------------------------------------------------------------------
>
> Key: KNOX-3474
> URL: https://issues.apache.org/jira/browse/KNOX-3474
> Project: Apache Knox
> Issue Type: Task
> Reporter: Arun Kumar M
> Priority: Major
>
> *Summary*
> Add upstream Knox support for the Spark 4 History Server Web UI (role:
> SPARK4HISTORYUI).
>
> *Motivation*
> Knox already supports spark3historyui (SPARK3HISTORYUI). Spark 4 deployments
> need an equivalent service definition so the History Server UI is reachable
> through Knox and linked correctly from YARN ResourceManager UI.
>
> *Implementation*
> Base on existing spark3historyui/3.0.0/ pattern:
>
> | Item | Value |
> |------|-------|
> | Service role | SPARK4HISTORYUI |
> | Service name / URL path | spark4history |
> | Metadata type | UI |
> | shortDesc | Spark 4 History Server Web UI |
> | CM discovery | Spark4HistoryUIServiceModelGenerator (extend
> SparkHistoryUIServiceModelGenerator) |
> | CM service type | SPARK4_ON_YARN (align with CM Spark 4 service type) |
> | CM role type | SPARK4_YARN_HISTORY_SERVER (align with CM Spark 4 role type)
> |
>
> *Scope*
> *1. Service definition*
> - gateway-service-definitions/services/spark4historyui/3.0.0/service.xml
> - gateway-service-definitions/services/spark4historyui/3.0.0/rewrite.xml
> - Inbound/outbound rewrite rules for /spark4history/, /history/, jobs pages,
> SSO header filters
>
> *2. Cloudera Manager discovery*
> - Spark4HistoryUIServiceModelGenerator.java + unit tests
> - META-INF/services ServiceModelGenerator registration
> - ClouderaManagerServiceDiscoveryTest (HTTP and SSL port variants)
>
> *3. YARN UI cross-links (required for RM → Spark 4 History navigation)*
> - gateway-service-definitions/services/yarnui/2.7.0/rewrite.xml
> - Add spark4history redirect rule parallel to existing spark3history rule
> - Pattern: *://*:*/history/\{**}?spark4=true?\{**} →
> \{$frontend[url]}/spark4history/history/\{**}?\{**}
> - gateway-service-definitions/services/yarnuiv2/3.0.0/rewrite.xml
> - Same spark4history rule as above
>
> *4. Admin / configuration*
> - gateway-admin-ui: add SPARK4HISTORYUI to service role list
> - Homepage icon for Spark 4 History Server (if icon metadata support is
> included)
>
> *Out of scope*
> - Spark History Server SPNEGO / authentication configuration
> - Spark 4 parcel installation issues
>
> *Acceptance criteria*
> - /spark4history/** routes to Spark 4 History Server backend through Knox
> - CM discovery resolves SPARK4HISTORYUI URLs (HTTP and HTTPS)
> - YARN RM UI links with ?spark4=true route to Knox spark4history path
> - Service tile appears on Knox Homepage
> - Unit tests pass
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
