[
https://issues.apache.org/jira/browse/HADOOP-19972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18138116#comment-18138116
]
ASF GitHub Bot commented on HADOOP-19972:
-----------------------------------------
hadoop-yetus commented on PR #8704:
URL: https://github.com/apache/hadoop/pull/8704#issuecomment-6638745373
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 0s | | Docker mode activated. |
| -1 :x: | patch | 0m 22s | |
https://github.com/apache/hadoop/pull/8704 does not apply to trunk. Rebase
required? Wrong Branch? See
https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help.
|
| Subsystem | Report/Notes |
|----------:|:-------------|
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8704/33/console |
| versions | git=2.34.1 |
| Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
This message was automatically generated.
> Upgrade to Jetty 12 ee8
> -----------------------
>
> Key: HADOOP-19972
> URL: https://issues.apache.org/jira/browse/HADOOP-19972
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: build, common
> Reporter: Jose Luis López
> Assignee: Jose Luis López
> Priority: Major
> Labels: pull-request-available
>
> Move Hadoop's web servers from Jetty 9.4.58 to Jetty 12.1.14 on the ee8
> environment, keeping {{javax.servlet}}. This issue is the port only:
> everything that also works on Jetty 9.4 lands first, in the issues under
> *Depends on*.
> *Scope*:
> * hadoop-project/pom.xml: set jetty.version to 12.1.14. Replace the managed
> jetty-servlet, jetty-webapp, javax-websocket-server-impl and websocket-client
> with jetty-ee8-nested, jetty-ee8-servlet, jetty-ee8-webapp,
> jetty-ee8-websocket-jetty-server (excluding jetty-ee8-annotations, which
> brings the banned asm) and jetty-ee8-websocket-jetty-client. Drop
> jetty-servlet-tester, which nothing uses.
> * Rename the Jetty entries in the module poms: hadoop-auth, hadoop-common,
> hadoop-kms, hadoop-hdfs-httpfs, hadoop-yarn-client, hadoop-yarn-common,
> hadoop-yarn-services-api, the ApplicationHistoryServer, the NodeManager, the
> ResourceManager, the Router and timelineservice-hbase-tests. hadoop-sls also
> declares jetty-ee8-nested, and the NodeManager declares the WebSocket client
> for its tests.
> * hadoop-client-runtime and hadoop-client-minicluster: replace the Jetty 9.4
> coordinates in the exclusions and filters with the ee8 and
> jetty-websocket-core ones. The jetty-util exclusions of HADOOP-20014 stay.
> * LICENSE-binary and NOTICE-binary: the Jetty 12.1.14 artifacts in place of
> the 9.4 ones; drop javax.websocket-api.
> * HttpServer2: ee8 classes; Handler.Sequence in place of HandlerCollection;
> Server#setRequestLog in place of RequestLogHandler; a core handler for
> addHandlerAtFront; a /logs context that starts when hadoop.log.dir is
> missing; a connector that can reopen a closed listener; the two settings
> below.
> * HttpServer2Metrics: read Jetty 12's handle counters, convert nanoseconds to
> milliseconds, publish the five async metrics as 0.
> * Import changes to ee8 in the common servlets, McpHttpServer,
> ApplicationHistoryServer, ResourceManager, Router, ApiServerWebApp, SLSWebApp
> and the tests.
> * WebSocket: the client and NodeManager move to the ee8 WebSocket API.
> WebApps.Builder gets withServerConfigurer, so the NodeManager can install the
> WebSocket components before its context starts.
> * The YARN application catalog, only if it is still in the tree: exclude the
> ee8 artifacts from its four Hadoop dependencies, so its war keeps Jetty 9.4.
> * Tests: TestHttpServer, TestHttpServer2Metrics, TestSSLHttpServer,
> TestPseudoAuthenticator, TestCheckpoint and TestNMContainerWebSocket follow
> Jetty 12's behaviour.
> *Why*: Jetty 9.4 is end of life and its CVEs get no open-source fix
> (HADOOP-19910). Jetty 10 and 11 are end of life too, and Jetty 11 and later
> otherwise require {{jakarta.servlet}}. Jetty 12's ee8 environment still
> serves {{javax.servlet}}, so Hadoop reaches a supported Jetty without the
> namespace break. ee8 is a stage, not the destination: the port to Jetty 12's
> API happens here once, and HADOOP-19395 is then only the namespace change.
> *Incompatible changes*. The reason phrase alone makes this incompatible. This
> is the release note:
> # *No custom HTTP reason phrase.* The phrase is always the standard text for
> the status: "Unauthorized", not "Authentication required". Since
> HADOOP-20003, HADOOP-20004 and HADOOP-20005 the message is also in the
> response body, and this release's clients read it there. A client that reads
> only the phrase loses the message, and an older KMS client no longer
> re-authenticates on a 403 from the KMS, because it matches "Invalid
> signature" and "Anonymous requests are disallowed" against the phrase.
> # *Stricter URI parsing.* {{%2F}}, {{%2E}} segments and malformed encodings
> are refused with 400. {{hadoop.http.uri.compliance.violations}} controls what
> is allowed through; adding {{AMBIGUOUS_PATH_SEPARATOR}} accepts {{%2F}}
> again. The default keeps paths that Hadoop needs, and
> {{SUSPICIOUS_PATH_CHARACTERS}} in it also admits the encoded control
> characters {{%00}}, {{%09}}, {{%0A}}, {{%0D}} and {{%7F}}, as Jetty 9.4 did.
> # *TLS renegotiation is refused.* A client's TLS 1.2 renegotiation closes the
> connection. {{hadoop.http.ssl.renegotiation.allowed=true}} restores 9.4's
> behaviour.
> # *Metrics.* The five async HTTP metrics read 0, because Jetty 12 cannot
> count them. The {{dispatched*}} metrics now read Jetty 12's handle counters.
> All names, and the milliseconds, are unchanged.
> # *A response shorter than its Content-Length fails in the server.* The
> servlet's write fails instead of a truncated body reaching the client, so the
> error is reported on the other side: a short fsimage transfer now fails on
> the NameNode.
> # *Jetty's own output differs:* the error-page markup, {{charset=utf-8}} and
> {{Vary}} on static files, a {{Date}} header on error responses, the default
> acceptor count, and a context path without its trailing slash answering 301
> with a relative {{Location}} where 9.4 sent 302 with an absolute one.
> # *Internal API and dependencies.* {{InterfaceAudience.Private}} and
> {{LimitedPrivate}} classes that name Jetty types now name the ee8 ones:
> {{HttpServer2}} ({{addContext}}, {{defineFilter}}, {{getWebAppContext}},
> {{addDefaultApps}}), {{WebApps}} and {{ContainerShellWebSocket}}.
> {{addHandlerAtFront}} takes a core handler, so an ee8 context goes through
> {{getCoreContextHandler()}}. Code calling these must be rebuilt. Jetty's
> artifacts move to {{org.eclipse.jetty.ee8}}, so downstream exclusions need
> the new coordinates. Public {{javax.servlet}} extension points do not change;
> TestDownstreamServletCompatibility pins them.
> Everything else keeps 9.4's behaviour: {{/static}} lists no directory,
> {{/logs}} keeps its admin check when the log directory is missing,
> {{web.xml}} error pages work, a stopped listener can be reopened, the
> NodeManager's WebSocket upgrade works, and a {{Location}} set through
> {{sendRedirect}} stays absolute.
> *New settings*:
> ||Setting||Default||
> |{{hadoop.http.uri.compliance.violations}}|{{AMBIGUOUS_EMPTY_SEGMENT,AMBIGUOUS_PATH_ENCODING,SUSPICIOUS_PATH_CHARACTERS}}|
> |{{hadoop.http.ssl.renegotiation.allowed}}|{{false}}|
> *Downstream*:
> * Projects that use only the public API and {{javax.servlet}}: no rebuild, no
> namespace change.
> * Non-shaded artifacts: Jetty 12 and {{org.eclipse.jetty.ee8}} jars replace
> Jetty 9.4's on the classpath. A project pinning Jetty 9.4 next to Hadoop can
> no longer share one Jetty release with it.
> * hadoop-client-runtime and hadoop-client-minicluster: the relocated Jetty is
> 12. Relocated classes are not API.
> * The application catalog war keeps Jetty 9.4 through {{solr.jetty.version}}
> until HADOOP-19998, if HADOOP-20008 landed. If HADOOP-19998 landed first, the
> module is gone and no Jetty 9.4 artifact is left.
> *Depends on*: HADOOP-20012, HADOOP-20013, HADOOP-20014, HADOOP-20015,
> HADOOP-20016, HADOOP-19970, HADOOP-20049, HADOOP-20003, HADOOP-20004,
> HADOOP-20005, HADOOP-20006, HADOOP-20007, and HADOOP-20008 or HADOOP-19998.
> *Not in scope*: {{jakarta.servlet}}, Jersey upgrades and ee9, ee10 or ee11:
> HADOOP-19395. Guice 6 (HADOOP-19999) and jsr311-api (HADOOP-20000) are not
> needed by this issue. HADOOP-19971 belongs to HADOOP-19395, and this issue
> does not depend on it. Requires Java 17, so it cannot go to branch-3.3 or
> branch-3.4.
> *CI*: touches hadoop-project, so a full build, including
> hadoop-client-check-invariants, hadoop-client-check-test-invariants and
> hadoop-client-integration-tests.
> *PR*: https://github.com/apache/hadoop/pull/8704 (draft, rebuilt on trunk and
> stacked on its prerequisites; review the last 6 commits). The prerequisite
> commits are the pre-2026-11-04 split and do not equal the current 20015,
> 20016 and 19970 PRs: `6f962fe1de6` mixes the old 20015 pom scope with a Java
> change; together with `6c7755527b7` and `78b813d4d6a` it covers 11 of the 12
> files in #8893 (`TestLogAggregationService` is in the port commit
> `dffea5f770e`), `2943da135c1` and `6f962fe1de6` cover 9 of the 13 poms in
> #8894 (hdfs, jobclient, NM and web-proxy are in the 19970/19972 commits), and
> `8d43e65e786` is the old `jetty-servlet-api` 4.0.9, overridden by
> `dd8150590cc`. They need replacing when the branch is rebuilt.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
