Huang Kuan Hao created HDDS-16358:
-------------------------------------
Summary: Gate the S3 derived-key piggyback on an
OzoneManagerVersion
Key: HDDS-16358
URL: https://issues.apache.org/jira/browse/HDDS-16358
Project: Apache Ozone
Issue Type: Sub-task
Reporter: Huang Kuan Hao
Assignee: Huang Kuan Hao
HDDS-15140 added the `derivedKeyPiggyBacking` field (field 3) to `KeyArgs` and
the matching
`ClientProtocol`/`RpcClient` overloads, but did not add a corresponding
`OzoneManagerVersion`
constant. `RpcClient.createKey(..., derivedKeyPiggyBacking)` sets the flag
unconditionally,
unlike the adjacent `rewriteKey`, which checks
`omVersion.compareTo(OzoneManagerVersion.ATOMIC_REWRITE_KEY) < 0` first.
That was harmless while nothing depended on the response. HDDS-15141 makes it
load-bearing:
`EndpointBase.attachChunkValidator` treats a missing derived key as a
server-side anomaly and
rejects the request in secure mode, rather than storing the payload unverified.
The consequence is a rolling-upgrade window. With
`ozone.security.enabled=true`, an S3 Gateway
that has HDDS-15141 talking to an OM that predates HDDS-15140 will:
1. send `derivedKeyPiggyBacking=true`,
2. have the old OM silently ignore the unknown optional field and return no
derived key,
3. fail the request with InternalError.
Every `x-amz-content-sha256: STREAMING-AWS4-HMAC-SHA256-PAYLOAD` upload fails
until OM is
upgraded. Both HDDS-15140 and HDDS-15141 target 2.3.0, so this only affects a
mixed 2.2.x OM /
2.3.0 S3G deployment, not an all-2.3.0 one.
The hard failure itself is deliberate and should stay — in secure mode, storing
a payload whose
chunk signatures cannot be verified is the worse outcome. What is missing is
the capability
check that would let S3G tell "this OM is too old for chunk verification" apart
from "this OM
should have sent a key and did not".
## Proposed fix （I guess）
1. Add an `OzoneManagerVersion` constant for the derived-key piggyback (next
free ordinal, 13),
describing the OM build that introduced `KeyArgs.derivedKeyPiggyBacking`.
2. Gate the flag in `RpcClient` so it is only requested when the peer OM
supports it.
3. Surface that capability to the S3 Gateway so
`EndpointBase.getS3ChunkInputStreamInfo` can
decline to opt into verification against an older OM, instead of opting in
and then failing
in `attachChunkValidator`.
4. Add a test covering the mixed-version path.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
