Arun Kumar M created KNOX-3507:
----------------------------------
Summary: Wire LivyHaDispatch for LIVYSERVER, LIVY_FOR_SPARK3, and
LIVY_FOR_SPARK4
Key: KNOX-3507
URL: https://issues.apache.org/jira/browse/KNOX-3507
Project: Apache Knox
Issue Type: Task
Reporter: Arun Kumar M
h2. Summary
Restore baseline Livy HA dispatch wiring that was omitted when KNOX-3472 and
KNOX-3473 landed the Livy for Spark 3/4 Knox service definitions. This aligns
upstream Apache Knox with internal Gerrit change 214416 (CDPD-45165): baseline
\{{LivyHaDispatch}}, shared \{{LivyHttpServletRequest}}, and \{{ha-classname}}
on the Livy-related service descriptors.
h2. Background
* [KNOX-3472|https://issues.apache.org/jira/browse/KNOX-3472] /
[#1411|https://github.com/apache/knox/pull/1411] added \{{livy_for_spark3}}
(service definition, CM discovery, rewrites).
* [KNOX-3473|https://issues.apache.org/jira/browse/KNOX-3473] /
[#1412|https://github.com/apache/knox/pull/1412] added \{{livy_for_spark4}}.
* Those PRs used \{{LivyDispatch}} only and did not wire
\{{org.apache.knox.gateway.ha.dispatch.LivyHaDispatch}}.
* Per-instance homepage routing (\{{?host=}}, sticky cookie, multi-tile UI) is
tracked separately in
[KNOX-3479|https://issues.apache.org/jira/browse/KNOX-3479] /
[#1420|https://github.com/apache/knox/pull/1420].
h2. Proposed changes
# Add \{{gateway-provider-ha/.../LivyHaDispatch.java}} — Livy-specific HA (HTTP
307 handling, failover) and Livy POST handling via \{{LivyHttpServletRequest}}.
# Extract \{{LivyHttpServletRequest}} to \{{gateway-service-livy}} (shared by
\{{LivyDispatch}} and \{{LivyHaDispatch}}).
# Add \{{gateway-service-livy}} dependency in \{{gateway-provider-ha/pom.xml}}.
# Set \{{ha-classname="org.apache.knox.gateway.ha.dispatch.LivyHaDispatch"}} on:
#* \{{services/livy/0.4.0/service.xml}} (LIVYSERVER)
#* \{{services/livy_for_spark3/0.6.0/service.xml}}
#* \{{services/livy_for_spark4/1.0.0/service.xml}}
h2. Out of scope (KNOX-3479)
* Per-instance backend selection (\{{?host=}}, Referer, \{{KNOX_LIVY_BACKEND}}
cookie)
* Knox Homepage tile hostname disambiguation
* Changes to \{{livy_for_spark3}} / \{{livy_for_spark4}} homepage \{{context}}
with \{{?host={{BACKEND_HOST}}}}
h2. Upstream PR
[apache/knox#1443|https://github.com/apache/knox/pull/1443] — branch
\{{KNOX-livy-ha-dispatch-wiring}}
h2. Testing
{code}
mvn -pl gateway-service-livy,gateway-provider-ha,gateway-service-definitions
-am test -DskipITs
{code}
Manual (optional): topology with HA-enabled Livy / Livy for Spark 3/4 URLs;
confirm default HA failover behavior for batch/session POST and 307 handling.
h2. Downstream
After merge, backport to CDH \{{knox}} (coordinate with CDPD-99323 / #758 and
avoid duplicating Gabor’s #796 if full backport covers Spark 4 wiring).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
