[
https://issues.apache.org/jira/browse/HDDS-8280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Yarovoy updated HDDS-8280:
---------------------------------
Release Note:
Notable behavioral changes
Admin servlets. hadoop-common's org.apache.hadoop.conf.ConfServlet and
org.apache.hadoop.jmx.JMXJsonServlet both route their admin-access check through
org.apache.hadoop.http.HttpServer2, which references
org.eclipse.jetty.servlet.ServletContextHandler,
org.eclipse.jetty.webapp.WebAppContext and FilterHolder/FilterMapping — all
removed or moved
in Jetty 12. Those classes cannot load at all on a Jetty 12 classpath, so /jmx
and /conf would
answer 500 with a NoClassDefFoundError. This is unrelated to the EE
environment. /conf is
switched to Ozone's existing HddsConfServlet, and a new HddsJMXJsonServlet
covers /jmx.
URI compliance. Jetty 12 rejects ambiguous URIs (empty // segments, %25,
encoded path
separators, suspicious path characters) that 9.4 passed through. S3 object keys
and WebHDFS paths
need them. A new opt-in Builder.allowAmbiguousUri(boolean) installs an OZONE
UriCompliance
mode allowing exactly those four violations — and, unlike Jetty's LEGACY mode,
not
%2e%2e path traversal, UTF-16 %u encodings, truncated UTF-8 or userinfo. S3
Gateway and
httpfs opt in; every other endpoint keeps Jetty's strict default.
XML content type. EE8's Response.setContentType appends ;charset= once
_encodingFrom
has moved past NOT_SET, and QuotingInputFilter promotes it on every request,
whereas 9.4
reset it — so a bare application/xml would ship as
application/xml;charset=utf-8. A new
S3ContentTypeFilter resets the encoding to NOT_SET to preserve the S3 wire
format.
Reason phrases. Jetty 12 discards custom HTTP reason phrases at the core level,
so
S3SecretManagementEndpoint moves its error code into a text/plain entity body.
TLS SNI. Jetty 12 defaults SecureRequestCustomizer#sniHostCheck to true, which
breaks
Ozone's hostname-agnostic TLS endpoints; it is constructed with false.
Base resources. Core ContextHandler.doStart() now throws for a missing base
directory,
where 9.4 tolerated it. The log directory is created best-effort and /logs is
skipped rather
than failing startup if unavailable; /static is served only when its base
resource exists.
Weld / CDI. Weld 3.1.9's JettyLegacyContainer detects on
org.eclipse.jetty.util.Decorator,
which still exists in jetty-util 12, so it is selected — then
LegacyWeldDecorator.process()
touches the deleted org.eclipse.jetty.servlet.ServletContextHandler and throws
NoClassDefFoundError. Because that is an Error, Weld's catch (Exception) does
not catch it
and S3 Gateway context startup fails. Fixed with Jetty's canonical
jetty-ee8-cdi module and a
CdiDecoratingListener, which makes Weld's probe match JettyContainer first.
The listener is installed on each of the three S3 Gateway contexts that
register the Weld listener
in their descriptor, guarded by isEnabled(): BaseHttpServer builds no
HttpServer2 at all for a
disabled server, and ozone.s3g.sts.http.enabled is false by default, so an
unguarded call would
NPE and kill the whole gateway at startup. BaseHttpServer.isEnabled() is
widened from private to
protected for that check.
httpfs descriptors. <url-pattern>*</url-pattern> is illegal per the Servlet
spec and is
corrected to /* (5 filter-mappings in each of the two descriptors).
hadoop-kms removed from integration-test. MiniKMS embeds an unshaded Jetty 9
Server and
cannot coexist with a Jetty 12 runtime. TestOzoneAtRestEncryption and
TestOzoneShellHA now
pre-seed a JavaKeyStoreProvider jceks before cluster start instead. The native
secure-KMS path
is followed up in HDDS-16424.
> Upgrade Jetty to 12
> -------------------
>
> Key: HDDS-8280
> URL: https://issues.apache.org/jira/browse/HDDS-8280
> Project: Apache Ozone
> Issue Type: Task
> Reporter: Siyao Meng
> Assignee: Andrey Yarovoy
> Priority: Major
> Labels: pull-request-available
>
> Hurdles:
> 1. Upgrading jetty to 10.x or higher means ditching support for JDK 8. 10.x
> or 11.x requires JDK 11 to compile and run. 12.x requires JDK 17. See this:
> https://www.eclipse.org/jetty/download.php#version-table
> 2. Community Support for Jetty 9.x has ended. There doesn't seem to have a
> plan to support JDK 17 in Jetty 9.4.x:
> https://github.com/eclipse/jetty.project/issues/9196. Unless there is another
> fork.
> 3. Even if ditching JDK 8 is tolerable, some servlets
> (AdminAuthorizedServlet, HttpServer2) have to be manually migrated to
> accommodate new jetty. Here I attempted to compile with jetty 11.0.14 + Zulu
> JDK 17:
> {code}
> [ERROR] COMPILATION ERROR :
> [INFO] -------------------------------------------------------------
> [ERROR]
> /Users/smeng/repo/ozone-master/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/http/AdminAuthorizedServlet.java:[36,3]
> method does not override or implement a method from a supertype
> [ERROR]
> /Users/smeng/repo/ozone-master/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/http/AdminAuthorizedServlet.java:[40,61]
> incompatible types: jakarta.servlet.ServletContext cannot be converted to
> javax.servlet.ServletContext
> [ERROR]
> /Users/smeng/repo/ozone-master/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/http/AdminAuthorizedServlet.java:[43,19]
> incompatible types: javax.servlet.http.HttpServletRequest cannot be
> converted to jakarta.servlet.http.HttpServletRequest
> [ERROR]
> /Users/smeng/repo/ozone-master/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/http/HttpServer2.java:[575,69]
> incompatible types: org.eclipse.jetty.server.handler.ContextHandler.Context
> cannot be converted to javax.servlet.ServletContext
> [ERROR]
> /Users/smeng/repo/ozone-master/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/http/HttpServer2.java:[834,30]
> no suitable constructor found for
> ServletHolder(java.lang.Class<com.sun.jersey.spi.container.servlet.ServletContainer>)
> constructor
> org.eclipse.jetty.servlet.ServletHolder.ServletHolder(org.eclipse.jetty.servlet.Source)
> is not applicable
> (argument mismatch;
> java.lang.Class<com.sun.jersey.spi.container.servlet.ServletContainer> cannot
> be converted to org.eclipse.jetty.servlet.Source)
> constructor
> org.eclipse.jetty.servlet.ServletHolder.ServletHolder(jakarta.servlet.Servlet)
> is not applicable
> (argument mismatch;
> java.lang.Class<com.sun.jersey.spi.container.servlet.ServletContainer> cannot
> be converted to jakarta.servlet.Servlet)
> constructor
> org.eclipse.jetty.servlet.ServletHolder.ServletHolder(java.lang.Class<?
> extends jakarta.servlet.Servlet>) is not applicable
> (argument mismatch;
> java.lang.Class<com.sun.jersey.spi.container.servlet.ServletContainer> cannot
> be converted to java.lang.Class<? extends jakarta.servlet.Servlet>)
> [ERROR]
> /Users/smeng/repo/ozone-master/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/http/HttpServer2.java:[883,28]
> no suitable constructor found for ServletHolder(java.lang.Class<capture#1 of
> ? extends javax.servlet.http.HttpServlet>)
> constructor
> org.eclipse.jetty.servlet.ServletHolder.ServletHolder(org.eclipse.jetty.servlet.Source)
> is not applicable
> (argument mismatch; java.lang.Class<capture#1 of ? extends
> javax.servlet.http.HttpServlet> cannot be converted to
> org.eclipse.jetty.servlet.Source)
> constructor
> org.eclipse.jetty.servlet.ServletHolder.ServletHolder(jakarta.servlet.Servlet)
> is not applicable
> (argument mismatch; java.lang.Class<capture#1 of ? extends
> javax.servlet.http.HttpServlet> cannot be converted to
> jakarta.servlet.Servlet)
> constructor
> org.eclipse.jetty.servlet.ServletHolder.ServletHolder(java.lang.Class<?
> extends jakarta.servlet.Servlet>) is not applicable
> (argument mismatch; java.lang.Class<capture#1 of ? extends
> javax.servlet.http.HttpServlet> cannot be converted to java.lang.Class<?
> extends jakarta.servlet.Servlet>)
> [ERROR]
> /Users/smeng/repo/ozone-master/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/http/HttpServer2.java:[957,30]
> no suitable constructor found for ServletHolder(java.lang.Class<capture#2 of
> ? extends javax.servlet.http.HttpServlet>)
> constructor
> org.eclipse.jetty.servlet.ServletHolder.ServletHolder(org.eclipse.jetty.servlet.Source)
> is not applicable
> (argument mismatch; java.lang.Class<capture#2 of ? extends
> javax.servlet.http.HttpServlet> cannot be converted to
> org.eclipse.jetty.servlet.Source)
> constructor
> org.eclipse.jetty.servlet.ServletHolder.ServletHolder(jakarta.servlet.Servlet)
> is not applicable
> (argument mismatch; java.lang.Class<capture#2 of ? extends
> javax.servlet.http.HttpServlet> cannot be converted to
> jakarta.servlet.Servlet)
> constructor
> org.eclipse.jetty.servlet.ServletHolder.ServletHolder(java.lang.Class<?
> extends jakarta.servlet.Servlet>) is not applicable
> (argument mismatch; java.lang.Class<capture#2 of ? extends
> javax.servlet.http.HttpServlet> cannot be converted to java.lang.Class<?
> extends jakarta.servlet.Servlet>)
> [INFO] 7 errors
> {code}
> The last point could very well impact to Hadoop/HDFS's jetty usage as well.
> Though I have not seen anyone mentioning in it under HADOOP-17177 yet.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
