[
https://issues.apache.org/jira/browse/KNOX-3472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Kumar M updated KNOX-3472:
-------------------------------
Description:
Summary
Add upstream Knox support for the dedicated Livy for Spark 3 service (role:
LIVY_FOR_SPARK3), distinct from the existing generic "livy" service definition.
Motivation
Many deployments run a separate Livy for Spark 3 instance alongside the legacy
Livy service. Knox currently exposes only the generic livy service;
livy_for_spark3 is required for correct proxy routing, Cloudera Manager
discovery, and Knox Homepage tiles.
Scope
1. Service definition (gateway-service-definitions)
- services/livy_for_spark3/0.6.0/service.xml
- services/livy_for_spark3/0.6.0/rewrite.xml
- Metadata: type=API_AND_UI, context=/livy_for_spark3/, shortDesc, description
- Dispatch: org.apache.knox.gateway.livy.LivyDispatch (reuse existing
gateway-service-livy module)
- Inbound/outbound rewrite rules for REST API (/v1), UI (/ui), YARN proxy
links, and container log links
2. Cloudera Manager discovery (gateway-discovery-cm)
- LivyForSpark3ServiceModelGenerator.java
- SERVICE / SERVICE_TYPE: LIVY_FOR_SPARK3
- ROLE_TYPE: LIVY_SERVER_FOR_SPARK3
- Register in
META-INF/services/org.apache.knox.gateway.topology.discovery.cm.ServiceModelGenerator
- LivyForSpark3ServiceModelGeneratorTest.java
- ClouderaManagerServiceDiscoveryTest coverage for LIVY_FOR_SPARK3 URL discovery
3. Admin / configuration
- gateway-admin-ui: add LIVY_FOR_SPARK3 to new-desc-wizard service role list
- gateway-release/home/conf/gateway-site.xml: add LIVY_FOR_SPARK3 to
gateway.xforwarded.header.context.append.servicename
4. Homepage icon (optional in same PR or follow-up)
- Add livyserver.png and iconFileName metadata support if not already present
upstream
Out of scope
- Per-instance HA routing (custom ha-classname, ?host= query param, sticky
cookies)
- Knox Homepage JavaScript tile-label patching
Acceptance criteria
- livy_for_spark3 service definition loads and routes /livy_for_spark3/**
through Knox
- CM discovery resolves LIVY_FOR_SPARK3 backend URLs for LIVY_SERVER_FOR_SPARK3
roles
- Service appears in Knox Admin UI descriptor wizard and Knox Homepage (UI +
API topology)
- Unit tests pass for model generator and CM discovery
was:
Add upstream Knox support for the Livy for Spark 3 service (role:
LIVY_FOR_SPARK3).
Motivation:
CDP clusters run a dedicated Livy for Spark 3 service distinct from the legacy
Livy service.
Knox currently only has a generic "livy" service definition; livy_for_spark3 is
missing upstream.
Scope:
- gateway-service-definitions/.../livy_for_spark3/0.6.0/service.xml +
rewrite.xml
- gateway-discovery-cm/.../LivyForSpark3ServiceModelGenerator.java + tests
- gateway-admin-ui service role registration
- gateway-release/home/conf/gateway-site.xml (service role list)
- Homepage icon: livyserver.png
Source: Port from CDH/knox (CDPD-15922, CDPD-77736 metadata/icon changes).
Exclude CDP-specific HA dispatch (LivyHaDispatch) — tracked separately in
CDPD-99323.
> Add Knox service definition for Livy for Spark 3 (livy_for_spark3)
> ------------------------------------------------------------------
>
> Key: KNOX-3472
> URL: https://issues.apache.org/jira/browse/KNOX-3472
> Project: Apache Knox
> Issue Type: Task
> Reporter: Arun Kumar M
> Priority: Major
>
> Summary
> Add upstream Knox support for the dedicated Livy for Spark 3 service (role:
> LIVY_FOR_SPARK3), distinct from the existing generic "livy" service
> definition.
>
> Motivation
> Many deployments run a separate Livy for Spark 3 instance alongside the
> legacy Livy service. Knox currently exposes only the generic livy service;
> livy_for_spark3 is required for correct proxy routing, Cloudera Manager
> discovery, and Knox Homepage tiles.
>
> Scope
> 1. Service definition (gateway-service-definitions)
> - services/livy_for_spark3/0.6.0/service.xml
> - services/livy_for_spark3/0.6.0/rewrite.xml
> - Metadata: type=API_AND_UI, context=/livy_for_spark3/, shortDesc, description
> - Dispatch: org.apache.knox.gateway.livy.LivyDispatch (reuse existing
> gateway-service-livy module)
> - Inbound/outbound rewrite rules for REST API (/v1), UI (/ui), YARN proxy
> links, and container log links
>
> 2. Cloudera Manager discovery (gateway-discovery-cm)
> - LivyForSpark3ServiceModelGenerator.java
> - SERVICE / SERVICE_TYPE: LIVY_FOR_SPARK3
> - ROLE_TYPE: LIVY_SERVER_FOR_SPARK3
> - Register in
> META-INF/services/org.apache.knox.gateway.topology.discovery.cm.ServiceModelGenerator
> - LivyForSpark3ServiceModelGeneratorTest.java
> - ClouderaManagerServiceDiscoveryTest coverage for LIVY_FOR_SPARK3 URL
> discovery
>
> 3. Admin / configuration
> - gateway-admin-ui: add LIVY_FOR_SPARK3 to new-desc-wizard service role list
> - gateway-release/home/conf/gateway-site.xml: add LIVY_FOR_SPARK3 to
> gateway.xforwarded.header.context.append.servicename
>
> 4. Homepage icon (optional in same PR or follow-up)
> - Add livyserver.png and iconFileName metadata support if not already present
> upstream
>
> Out of scope
> - Per-instance HA routing (custom ha-classname, ?host= query param, sticky
> cookies)
> - Knox Homepage JavaScript tile-label patching
>
> Acceptance criteria
> - livy_for_spark3 service definition loads and routes /livy_for_spark3/**
> through Knox
> - CM discovery resolves LIVY_FOR_SPARK3 backend URLs for
> LIVY_SERVER_FOR_SPARK3 roles
> - Service appears in Knox Admin UI descriptor wizard and Knox Homepage (UI +
> API topology)
> - Unit tests pass for model generator and CM discovery
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
