jojochuang commented on code in PR #11367:
URL: https://github.com/apache/ozone/pull/11367#discussion_r4141294822
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/http/HttpServer2.java:
##########
@@ -172,6 +172,29 @@ public final class HttpServer2 implements FilterContainer {
private static final int HTTP_IDLE_TIMEOUT_MS_DEFAULT = 60000;
private static final String HTTP_TEMP_DIR_KEY = "hadoop.http.temp.dir";
+ /**
+ * URI compliance mode used when {@code allowAmbiguousUri} is set. S3 object
+ * keys and WebHDFS paths legitimately contain empty path segments ("//"),
+ * percent encodings ("%25"), encoded path separators, and after decoding a
+ * backslash or other suspicious character (DEL and C0 controls), all of
which
+ * Jetty 12 rejects with 400 by default. Jetty 9.4 (pre-migration) had no
such
+ * check and passed these through, so relax those four violations to keep
that
+ * behavior; a decoded backslash or control byte is opaque key/path data in
+ * Ozone (which uses only "/" as a separator), so it does not open path
+ * traversal. Rather than Jetty's broad LEGACY mode -- which would also
+ * re-admit %2e/%2e%2e path traversal, UTF-16 and truncated UTF-8 encodings
and
+ * userinfo on the internet-facing S3 Gateway and HttpFS -- relax only those
+ * violations the use case needs. Genuinely illegal (unencoded) URI
characters
+ * such as "[" and "]" remain rejected, since conforming clients
percent-encode
+ * them.
+ */
+ private static final UriCompliance OZONE_AMBIGUOUS_URI_COMPLIANCE =
+ UriCompliance.DEFAULT.with("OZONE",
+ UriCompliance.Violation.AMBIGUOUS_EMPTY_SEGMENT,
+ UriCompliance.Violation.AMBIGUOUS_PATH_ENCODING,
+ UriCompliance.Violation.AMBIGUOUS_PATH_SEPARATOR,
+ UriCompliance.Violation.SUSPICIOUS_PATH_CHARACTERS);
Review Comment:
[P2] Preserve encoded dot segments in S3 object keys
`shouldAllowAmbiguousUri()` returning true selects the four explicitly
allowed violations here; it does not allow every ambiguity type.
`AMBIGUOUS_PATH_SEGMENT` is still excluded, so a request for key
`dir/./file.txt` encoded as `/bucket/dir/%2e/file.txt` receives HTTP 400 before
reaching the S3 endpoint. OBS buckets preserve these segments as part of the
key. AWS documents period-only segments and qualifying relative segments as
valid key components:
https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html
I reproduced the connector behavior with isolated JDK 21 wire probes: Jetty
9.4.58 accepts `/bucket/dir/%2e/file` and `/bucket/dir/%2e%2e/file` and
preserves their request URIs, whereas Jetty 12.0.38 with this patch's exact
relaxed compliance configuration returns 400. This was a connector-level
reproduction, not a full Ozone round-trip test.
Suggested fix: allow `UriCompliance.Violation.AMBIGUOUS_PATH_SEGMENT` in a
separate S3-specific compliance mode. Since the current relaxed mode is shared
with HttpFS, adding it directly to this constant would also change HttpFS
behavior; retain HttpFS's existing restrictions.
Please add PUT/GET/DELETE round-trip coverage through a backend gateway
directly, bypassing the test proxy, for encoded dot segments in keys such as
`dir/./file.txt` and `dir/../file.txt`. Verify that these remain distinct from
`dir/file.txt` and `file.txt`, respectively. This validates that admitting the
requests also preserves the key through Jetty/Jersey rather than normalizing it
during dispatch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
