[
https://issues.apache.org/jira/browse/HADOOP-20014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18137934#comment-18137934
]
ASF GitHub Bot commented on HADOOP-20014:
-----------------------------------------
joseluisll commented on PR #8892:
URL: https://github.com/apache/hadoop/pull/8892#issuecomment-6628887584
Thanks @pan3793 for the review.
`jetty-util` is already shaded and relocated, and downstream projects never
see it as a dependency. I built `hadoop-client-minicluster` on trunk and on
this PR and compared the jars:
| | trunk | this PR |
|---|---|---|
| relocated `jetty-util` classes
(`org/apache/hadoop/shaded/org/eclipse/jetty/util/`) | 377 | 377 |
| unrelocated `org/eclipse/jetty/` classes | 0 | 0 |
| Jetty or Jersey dependencies in the published pom | 0 | 0 |
The only differences between the two jars are the Jersey test container
(Jetty → JDK HTTP) and the relocated `jetty-continuation`, which came only with
the Jetty container and is used by neither Hadoop nor downstream projects.
So `<optional>true</optional>` here doesn't make `jetty-util` an optional
transitive dependency. It is how this pom bundles dependencies into the shaded
jar, the same as `hadoop-minicluster`, Jersey and the test-framework provider.
To your questions:
1. **Who uses it at runtime?** `HttpServer2` (`SslContextFactory`,
`QueuedThreadPool`, …), which serves the mini clusters' web UIs.
2. **Why declare it now?** It used to arrive only through
`jersey-test-framework-provider-jetty` → `jersey-container-jetty-http` →
`jetty-util`. Switching to `provider-jdk-http` removed that path, so it is now
declared directly, and the jar keeps the same 377 classes.
3. **Shade and relocate instead?** It already is.
I also checked the downstream projects that depend on
`hadoop-client-minicluster` (Spark, Accumulo, Kyuubi, Druid, Zeppelin, Phoenix
connectors, DataFusion Comet, Kylin). None of them excludes or relies on
`jetty-util` or a Jersey test provider coming from it.
> HADOOP - Run the Jersey tests on the JDK HTTP server instead of Jetty
> ---------------------------------------------------------------------
>
> Key: HADOOP-20014
> URL: https://issues.apache.org/jira/browse/HADOOP-20014
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: hadoop-common, test
> Reporter: Jose Luis López
> Assignee: Jose Luis López
> Priority: Minor
> Labels: pull-request-available
>
> *Goal*: run JerseyTest on the JDK's built-in HTTP server
> (jersey-test-framework-provider-jdk-http) instead of Jetty. Jersey 2 has no
> container for Jetty 12, so this has to happen before HADOOP-19972. Doing it
> now takes it out of that upgrade. Production HTTP is not touched.
> *Scope*:
> * hadoop-project/pom.xml: manage jersey-test-framework-provider-jdk-http in
> place of -jetty, and keep the servlet API exclusion on it.
> * The 12 module poms that use provider-jetty: switch the artifactId.
> * hadoop-client-minicluster/pom.xml:
> ** bundle the jdk-http provider instead of the jetty one;
> ** declare org.eclipse.jetty:jetty-util directly, as
> <optional>true</optional>. Today the minicluster's only copy arrives through
> provider-jetty -> jersey-container-jetty-http, because its other dependencies
> exclude jetty-util as "in client". Without this, the switch would leave
> org.eclipse.jetty.util in neither shaded jar;
> ** add com/sun/net/httpserver/* and com/sun/net/httpserver/**/* to the com/
> relocation excludes. These are JDK classes and must not be relocated.
> * LICENSE-binary: jersey-test-framework-provider-jetty ->
> jersey-test-framework-provider-jdk-http, and add jersey-container-jdk-http,
> which the provider brings in and hadoop-client-minicluster bundles.
> * hadoop-client-runtime/pom.xml: *no change*. The jetty-util and
> jetty-continuation exclusions stay (YARN-11793), so hadoop-client-runtime
> ships no jetty-util; the minicluster ships its own (review on PR #8699).
> *Downstream*:
> * hadoop-client-runtime: unchanged, same jar contents.
> * hadoop-client-minicluster: keeps jetty-util. It loses the relocated
> jetty-continuation and jersey-container-jetty-http classes and gains the
> relocated jersey jdk-http container. All of these are under
> org.apache.hadoop.shaded and are not API.
> *Impact on Jetty 9*: production is unchanged, and HttpServer2 stays on Jetty
> 9.4. In the tests, JerseyTest no longer runs on Jetty. No test uses a servlet
> deployment. 58 of the 60 JerseyTest classes bind
> HttpServletRequest/HttpServletResponse mocks. JerseyTestBase is a base class,
> and the resourceestimator resource uses no servlet types.
> *Side effect*: removes the unmanaged jetty-continuation 9.4.55 that
> provider-jetty brought in.
> *CI*: touches hadoop-project, so a full build.
> *PR*: https://github.com/apache/hadoop/pull/8892
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
