Chung-En Lee created HDDS-16531:
-----------------------------------
Summary: Support direct S3 upload via Presigned PUT to DataNodes
Key: HDDS-16531
URL: https://issues.apache.org/jira/browse/HDDS-16531
Project: Apache Ozone
Issue Type: Improvement
Reporter: Chung-En Lee
h5. Motivation
In current Ozone deployments, all S3 traffic (both metadata and payload)
traverses the S3 Gateway (S3G). S3G acts as an active reverse-proxy,
translating HTTP streams into internal Ozone client RPC/gRPC/Ratis calls.
This architecture has major drawbacks for write-intensive workloads:
# {*}Network Double-Hop{*}: Client -> S3G -> DataNodes, making S3G instances a
massive network and CPU bottleneck.
# {*}Unnecessary Proxy Overhead{*}: S3G must buffer and manage streaming
chunks, increasing latency and memory pressure.
# {*}Deviation from Modern Object Storage Pattern{*}: Standard S3 Presigned
PUT is intended to offload data transmission directly to storage nodes after
obtaining authorization from the control plane.
h5. Proposed Architecture: Decoupled S3 Data Path
We propose decoupling S3G into a pure *Control Plane* (Metadata & Signing) and
leveraging *DataNodes (or co-located Data Workers)* as the {*}Data Plane{*}:
{code:java}
1. Request Presigned URL
Client -------------------> S3G / OM (Control Plane)
<------------------- Returns Presigned URL (pointing to target DN)
2. Direct Payload Upload (HTTP PUT)
Client -----------------------------------------------------> Leader DataNode
(Data Plane)
│
├──> Replicate
via Ratis Pipeline
│ (Follower
DNs)
└──> Ack /
Commit {code}
h5. Key Technical Considerations
# {*}DataNode HTTP Ingress{*}:
** Introduce a lightweight HTTP/Netty ingress service on DataNodes (disabled
by default or on a dedicated port).
** It accepts HTTP PUT requests carrying pre-authenticated authorization
context.
# {*}Authentication & Token Encapsulation{*}:
** How S3G encodes Ozone credentials: S3G validates AWS SigV4 / credentials
during URL generation, requests block/container allocation from OM, and embeds
an Ozone Block Token & pipeline metadata into the Presigned URL.
** DN validates the embedded token before accepting the payload.
# {*}Ratis Bridge on Leader DataNode{*}:
** The Leader DN acts as the stream head, converting incoming HTTP payload
directly into Ratis log entries / zero-copy chunk writes across the allocated
pipeline.
# {*}Lifecycle & Multipart Upload{*}:
** For single PUT: Automatic OM commit upon stream completion, or explicit
finalization.
** For Multipart Upload: Individual parts uploaded directly to DNs;
{{CompleteMultipartUpload}} processed by S3G / OM as a metadata-only
transaction.
h5. Goal of this JIRA
Start an architectural discussion and gather feedback on:
* Feasibility of embedding an HTTP receiver into DataNode vs. a standalone
edge sidecar.
* Security model for validating presigned parameters on storage nodes.
* Preparation of a formal design doc for community review.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
