[
https://issues.apache.org/jira/browse/KNOX-3472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Kumar M updated KNOX-3472:
-------------------------------
Description:
Summary
Add upstream Knox support for the dedicated Livy for Spark 3 service (role:
LIVY_FOR_SPARK3), distinct from the existing generic "livy" service definition.
Motivation
Many deployments run a separate Livy for Spark 3 instance alongside the legacy
Livy service. Knox currently exposes only the generic livy service;
livy_for_spark3 is required for correct proxy routing, Cloudera Manager
discovery, and Knox Homepage tiles.
Scope
# *Service definition (gateway-service-definitions)*
** services/livy_for_spark3/0.6.0/service.xml
** services/livy_for_spark3/0.6.0/rewrite.xml
** Metadata: type=API_AND_UI, context=/livy_for_spark3/, shortDesc, description
** Dispatch: org.apache.knox.gateway.livy.LivyDispatch (reuse existing
gateway-service-livy module)
** Inbound/outbound rewrite rules for REST API (/v1), UI (/ui), YARN proxy
links, and container log links
# *Cloudera Manager discovery (gateway-discovery-cm)*
** LivyForSpark3ServiceModelGenerator.java
** SERVICE / SERVICE_TYPE: LIVY_FOR_SPARK3
** ROLE_TYPE: LIVY_SERVER_FOR_SPARK3
** Register in
META-INF/services/org.apache.knox.gateway.topology.discovery.cm.ServiceModelGenerator
** LivyForSpark3ServiceModelGeneratorTest.java
** ClouderaManagerServiceDiscoveryTest coverage for LIVY_FOR_SPARK3 URL
discovery
# *Admin / configuration*
** gateway-admin-ui: add LIVY_FOR_SPARK3 to new-desc-wizard service role list
** gateway-release/home/conf/gateway-site.xml: add LIVY_FOR_SPARK3 to
gateway.xforwarded.header.context.append.servicename
# *Homepage icon (optional in same PR or follow-up)*
## Add livyserver.png and iconFileName metadata support if not already present
upstream
*Out of scope*
- Per-instance HA routing (custom ha-classname, ?host= query param, sticky
cookies)
- Knox Homepage JavaScript tile-label patching
Acceptance criteria
- livy_for_spark3 service definition loads and routes /livy_for_spark3/**
through Knox
- CM discovery resolves LIVY_FOR_SPARK3 backend URLs for
LIVY_SERVER_FOR_SPARK3 roles
- Service appears in Knox Admin UI descriptor wizard and Knox Homepage (UI +
API topology)
- Unit tests pass for model generator and CM discovery
was:
Summary
Add upstream Knox support for the dedicated Livy for Spark 3 service (role:
LIVY_FOR_SPARK3), distinct from the existing generic "livy" service definition.
Motivation
Many deployments run a separate Livy for Spark 3 instance alongside the legacy
Livy service. Knox currently exposes only the generic livy service;
livy_for_spark3 is required for correct proxy routing, Cloudera Manager
discovery, and Knox Homepage tiles.
Scope
# *Service definition (gateway-service-definitions)*
** services/livy_for_spark3/0.6.0/service.xml
** services/livy_for_spark3/0.6.0/rewrite.xml
** Metadata: type=API_AND_UI, context=/livy_for_spark3/, shortDesc, description
** Dispatch: org.apache.knox.gateway.livy.LivyDispatch (reuse existing
gateway-service-livy module)
** Inbound/outbound rewrite rules for REST API (/v1), UI (/ui), YARN proxy
links, and container log links
# *Cloudera Manager discovery (gateway-discovery-cm)*
** LivyForSpark3ServiceModelGenerator.java
** SERVICE / SERVICE_TYPE: LIVY_FOR_SPARK3
** ROLE_TYPE: LIVY_SERVER_FOR_SPARK3
** Register in
META-INF/services/org.apache.knox.gateway.topology.discovery.cm.ServiceModelGenerator
** LivyForSpark3ServiceModelGeneratorTest.java
** ClouderaManagerServiceDiscoveryTest coverage for LIVY_FOR_SPARK3 URL
discovery
# *Admin / configuration*
** gateway-admin-ui: add LIVY_FOR_SPARK3 to new-desc-wizard service role list
** gateway-release/home/conf/gateway-site.xml: add LIVY_FOR_SPARK3 to
gateway.xforwarded.header.context.append.servicename
# *Homepage icon (optional in same PR or follow-up)*
## Add livyserver.png and iconFileName metadata support if not already present
upstream
*Out of scope*
- Per-instance HA routing (custom ha-classname, ?host= query param, sticky
cookies)
- Knox Homepage JavaScript tile-label patching
Acceptance criteria
- livy_for_spark3 service definition loads and routes /livy_for_spark3/**
through Knox
- CM discovery resolves LIVY_FOR_SPARK3 backend URLs for
LIVY_SERVER_FOR_SPARK3 roles
- Service appears in Knox Admin UI descriptor wizard and Knox Homepage (UI +
API topology)
- Unit tests pass for model generator and CM discovery
> Add Knox service definition for Livy for Spark 3 (livy_for_spark3)
> ------------------------------------------------------------------
>
> Key: KNOX-3472
> URL: https://issues.apache.org/jira/browse/KNOX-3472
> Project: Apache Knox
> Issue Type: Task
> Reporter: Arun Kumar M
> Priority: Major
>
> Summary
> Add upstream Knox support for the dedicated Livy for Spark 3 service (role:
> LIVY_FOR_SPARK3), distinct from the existing generic "livy" service
> definition.
>
> Motivation
> Many deployments run a separate Livy for Spark 3 instance alongside the
> legacy Livy service. Knox currently exposes only the generic livy service;
> livy_for_spark3 is required for correct proxy routing, Cloudera Manager
> discovery, and Knox Homepage tiles.
>
> Scope
> # *Service definition (gateway-service-definitions)*
> ** services/livy_for_spark3/0.6.0/service.xml
> ** services/livy_for_spark3/0.6.0/rewrite.xml
> ** Metadata: type=API_AND_UI, context=/livy_for_spark3/, shortDesc,
> description
> ** Dispatch: org.apache.knox.gateway.livy.LivyDispatch (reuse existing
> gateway-service-livy module)
> ** Inbound/outbound rewrite rules for REST API (/v1), UI (/ui), YARN proxy
> links, and container log links
> # *Cloudera Manager discovery (gateway-discovery-cm)*
> ** LivyForSpark3ServiceModelGenerator.java
> ** SERVICE / SERVICE_TYPE: LIVY_FOR_SPARK3
> ** ROLE_TYPE: LIVY_SERVER_FOR_SPARK3
> ** Register in
> META-INF/services/org.apache.knox.gateway.topology.discovery.cm.ServiceModelGenerator
> ** LivyForSpark3ServiceModelGeneratorTest.java
> ** ClouderaManagerServiceDiscoveryTest coverage for LIVY_FOR_SPARK3 URL
> discovery
> # *Admin / configuration*
> ** gateway-admin-ui: add LIVY_FOR_SPARK3 to new-desc-wizard service role list
> ** gateway-release/home/conf/gateway-site.xml: add LIVY_FOR_SPARK3 to
> gateway.xforwarded.header.context.append.servicename
> # *Homepage icon (optional in same PR or follow-up)*
> ## Add livyserver.png and iconFileName metadata support if not already
> present upstream
>
> *Out of scope*
> - Per-instance HA routing (custom ha-classname, ?host= query param, sticky
> cookies)
> - Knox Homepage JavaScript tile-label patching
> Acceptance criteria
> - livy_for_spark3 service definition loads and routes /livy_for_spark3/**
> through Knox
> - CM discovery resolves LIVY_FOR_SPARK3 backend URLs for
> LIVY_SERVER_FOR_SPARK3 roles
> - Service appears in Knox Admin UI descriptor wizard and Knox Homepage (UI +
> API topology)
> - Unit tests pass for model generator and CM discovery
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
