yandrey321 opened a new pull request, #12012:
URL: https://github.com/apache/ozone/pull/12012

   ## What changes were proposed in this pull request?
   
   **Problem.** `ozone-common` and `ozone-filesystem-common` declare 
`httpclient` and `httpcore` only for two classes, used in four places:
   - `URIBuilder` builds the `o3fs://` / `ofs://` URI in `BasicOzoneFileSystem` 
and `BasicRootedOzoneFileSystem`, and
     the OM DB checkpoint URL in `OMNodeDetails`;
   - `org.apache.http.ParseException` is thrown twice in `OFSPath`.
   
   Those declarations are why the shaded filesystem jar bundles both libraries: 
773 entries, about 0.97 MB compressed. The
   jar is shipped with every job that uses `-libjars` or `spark.jars`.
   
   **Implementation idea.** Use JDK classes instead, then remove the two 
dependency declarations:
   - **Filesystem URIs:** `new URI(scheme + "://" + authority)`. That is the 
string `URIBuilder` builds and parses, so
     the result is the same, including the `URISyntaxException` for an illegal 
authority.
   - **Checkpoint URL:** the 5-argument `URI` constructor, with the query built 
from two constant keys and booleans.
   - **`OFSPath`:** throws `IllegalArgumentException` with the same messages.
   - **Guard:** a `ban-httpcomponents` enforcer rule in 
`ozone-filesystem-shaded`.
   
   **Key decisions.**
   - **No IPv6 bracketing for the filesystem URIs.** `URIBuilder` brackets an 
unbracketed IPv6 literal, and the JDK
     parse doesn't. Both file systems reject unbracketed IPv6 authorities 
earlier in `initialize`, so the input never
     gets that far.
   - **String parse rather than the 5-argument constructor for the filesystem 
URIs.** `getAuthority()` returns the
     decoded authority. The multi-argument constructors would quote it, so 
`ofs://om%20x/` would be accepted instead of
     failing with "Invalid Ozone endpoint", and `ofs://om%2Fx/` would come out 
differently.
   - **The ban is enabled only in `ozone-filesystem-shaded`.** The unshaded 
client modules still get `httpclient` from
     `hadoop-common` / `hadoop-hdfs-client`, so a ban there would fail the 
build. The shaded module catches both ways
     back: a re-added declaration, and code that uses httpclient, because 
dependency analysis would then require a
     declaration.
   
   **Side effects.**
   - `OFSPath` throws `IllegalArgumentException` instead of 
`org.apache.http.ParseException`. Both are unchecked, and
     nothing catches the http one. The `catch (ParseException ...)` blocks in 
`TrashPolicyOzone`, `S3ConditionalRequest`
     and `ReconUtils` are `java.text.ParseException`.
   - Applications that depend on the unshaded `ozone-common` / 
`ozone-filesystem-common` artifacts no longer get httpclient
     from them; they still get it from Hadoop. All Ozone modules that use 
httpclient (`s3gateway`, `cli-shell`, `vapor`,
     integration tests) declare it themselves.
   - No wire, proto, config or on-disk changes.
   
   
   ## What is the link to the Apache JIRA
   
   https://issues.apache.org/jira/browse/HDDS-17778
   
   ## How was this patch tested?
   
   Unit + integration tests.
   
    **Jar contents:** `ozone-filesystem-hadoop3` no longer has 
`org/apache/ozone/shaded/org/apache/http`,
     `mozilla/public-suffix-list.txt` or 
`META-INF/maven/org.apache.httpcomponents`. Nothing else changed apart from
     that. It is now 69.2 MB on disk.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to