[
https://issues.apache.org/jira/browse/HADOOP-19998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jose Luis López updated HADOOP-19998:
-------------------------------------
Description:
Sub-task of HADOOP-19912.
h3. Problem
After HADOOP-19972, every Hadoop module runs on Jetty 12 except the YARN
application catalog ({{hadoop-yarn-applications-catalog}}, YARN-7129). It stays
on Jetty 9.4.58 through {{solr.jetty.version}}, because its Solr 8.11.2
dependencies are built against Jetty 9.4:
||Dependency||Scope||Jetty 9.4 use||
|{{solr-solrj}} 8.11.2|compile (ships in the war)|SolrJ 8 uses Jetty 9.4's
HTTP/2 client ({{http2-client}}, {{http2-http-client-transport}},
{{http2-common}}, {{http2-hpack}})|
|{{solr-core}}, {{solr-test-framework}} 8.11.2|test|{{EmbeddedSolrServer}}
loads Jetty 9.4 client classes|
This is the last Jetty 9.4 in the build. Jetty 9.4 is end of life, so it keeps
the module exposed to unfixed CVEs and blocks closing HADOOP-19912.
h3. Current state of the module
* The war runs on Tomcat in a Docker image, next to a Solr server; it reads and
writes the catalog through SolrJ's {{HttpSolrClient}}.
* The last feature change was in April 2019 (YARN-8530, YARN-9281); since then
it has only received build and dependency maintenance.
* The Docker image build is broken:
{{hadoop-yarn-applications-catalog-docker/Dockerfile}} downloads
{{https://downloads.apache.org/lucene/solr/8.11.2/solr-8.11.2.tgz}}, which
returns 404 (the file now lives only at archive.apache.org).
* It holds 66 javax EE references (JAX-RS 48, JAXB 15, servlet 3) that
HADOOP-19395 would otherwise have to migrate.
h3. Options
*A. Retire the module (proposed).*
Remove {{hadoop-yarn-applications-catalog}} (webapp and docker) from the build,
together with {{solr.version}}, {{solr.jetty.version}} and the Solr entries in
LICENSE-binary/NOTICE-binary. YARN services remain deployable through the YARN
services REST API and CLI. This needs agreement on the dev list and a release
note.
*B. Upgrade to Solr 10.*
* War: move to SolrJ 10.0.0, which targets Java 17 and has no Jetty dependency.
{{HttpSolrClient}} is gone in SolrJ 10; replace it with {{HttpJdkSolrClient}},
which uses the JDK HTTP client. The Jetty 9.4 {{http2-*}} dependencies go away.
* Tests: solr-core 10.0.0 needs Java 21 and brings Jetty 12.0, while Hadoop
builds on Java 17 with Jetty 12.1, so the tests cannot keep
{{EmbeddedSolrServer}}. Rewrite them against a mocked {{SolrClient}}.
* Docker image: move to a Solr 10 server, migrate the Solr configuration, and
fix the download URL.
Solr 9 is not an option: it uses Jetty 10.
h3. Done when
* No Jetty 9.4 artifact is left on any module classpath of the trunk build, and
{{solr.jetty.version}} is removed.
* With option B: the catalog builds, its tests pass on Java 17 and 21, and the
Docker image builds and starts.
> Move the YARN application catalog off Solr 8 and Jetty 9.4
> ----------------------------------------------------------
>
> Key: HADOOP-19998
> URL: https://issues.apache.org/jira/browse/HADOOP-19998
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: yarn
> Reporter: Jose Luis López
> Priority: Major
>
> Sub-task of HADOOP-19912.
> h3. Problem
> After HADOOP-19972, every Hadoop module runs on Jetty 12 except the YARN
> application catalog ({{hadoop-yarn-applications-catalog}}, YARN-7129). It
> stays on Jetty 9.4.58 through {{solr.jetty.version}}, because its Solr 8.11.2
> dependencies are built against Jetty 9.4:
> ||Dependency||Scope||Jetty 9.4 use||
> |{{solr-solrj}} 8.11.2|compile (ships in the war)|SolrJ 8 uses Jetty 9.4's
> HTTP/2 client ({{http2-client}}, {{http2-http-client-transport}},
> {{http2-common}}, {{http2-hpack}})|
> |{{solr-core}}, {{solr-test-framework}} 8.11.2|test|{{EmbeddedSolrServer}}
> loads Jetty 9.4 client classes|
> This is the last Jetty 9.4 in the build. Jetty 9.4 is end of life, so it
> keeps the module exposed to unfixed CVEs and blocks closing HADOOP-19912.
> h3. Current state of the module
> * The war runs on Tomcat in a Docker image, next to a Solr server; it reads
> and writes the catalog through SolrJ's {{HttpSolrClient}}.
> * The last feature change was in April 2019 (YARN-8530, YARN-9281); since
> then it has only received build and dependency maintenance.
> * The Docker image build is broken:
> {{hadoop-yarn-applications-catalog-docker/Dockerfile}} downloads
> {{https://downloads.apache.org/lucene/solr/8.11.2/solr-8.11.2.tgz}}, which
> returns 404 (the file now lives only at archive.apache.org).
> * It holds 66 javax EE references (JAX-RS 48, JAXB 15, servlet 3) that
> HADOOP-19395 would otherwise have to migrate.
> h3. Options
> *A. Retire the module (proposed).*
> Remove {{hadoop-yarn-applications-catalog}} (webapp and docker) from the
> build, together with {{solr.version}}, {{solr.jetty.version}} and the Solr
> entries in LICENSE-binary/NOTICE-binary. YARN services remain deployable
> through the YARN services REST API and CLI. This needs agreement on the dev
> list and a release note.
> *B. Upgrade to Solr 10.*
> * War: move to SolrJ 10.0.0, which targets Java 17 and has no Jetty
> dependency. {{HttpSolrClient}} is gone in SolrJ 10; replace it with
> {{HttpJdkSolrClient}}, which uses the JDK HTTP client. The Jetty 9.4
> {{http2-*}} dependencies go away.
> * Tests: solr-core 10.0.0 needs Java 21 and brings Jetty 12.0, while Hadoop
> builds on Java 17 with Jetty 12.1, so the tests cannot keep
> {{EmbeddedSolrServer}}. Rewrite them against a mocked {{SolrClient}}.
> * Docker image: move to a Solr 10 server, migrate the Solr configuration, and
> fix the download URL.
> Solr 9 is not an option: it uses Jetty 10.
> h3. Done when
> * No Jetty 9.4 artifact is left on any module classpath of the trunk build,
> and {{solr.jetty.version}} is removed.
> * With option B: the catalog builds, its tests pass on Java 17 and 21, and
> the Docker image builds and starts.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
