This is an automated email from the ASF dual-hosted git repository.
weichiu pushed a commit to branch HDDS-9225-website-v2
in repository https://gitbox.apache.org/repos/asf/ozone-site.git
The following commit(s) were added to refs/heads/HDDS-9225-website-v2 by this
push:
new 95cdd2ceb HDDS-14306. [Website v2] [Docs] [User Guide] S3 API (#194)
95cdd2ceb is described below
commit 95cdd2ceb4ef18c4c9523ef7516d2e82c8315e02
Author: Eric C. Ho <[email protected]>
AuthorDate: Thu Jan 8 09:39:03 2026 +0800
HDDS-14306. [Website v2] [Docs] [User Guide] S3 API (#194)
Co-authored-by: Wei-Chiu Chuang <[email protected]>
---
cspell.yaml | 3 +
docs/04-user-guide/01-client-interfaces/03-s3.md | 189 ++++++++++++++++++++++-
2 files changed, 191 insertions(+), 1 deletion(-)
diff --git a/cspell.yaml b/cspell.yaml
index 5059d94cc..7e323e375 100644
--- a/cspell.yaml
+++ b/cspell.yaml
@@ -156,3 +156,6 @@ words:
- Qihoo360
- Meituan
- Unicom
+# S3 Compatible third-party applications
+- Clickhouse
+- Teradata
diff --git a/docs/04-user-guide/01-client-interfaces/03-s3.md
b/docs/04-user-guide/01-client-interfaces/03-s3.md
index 9e60525e4..baa2dec55 100644
--- a/docs/04-user-guide/01-client-interfaces/03-s3.md
+++ b/docs/04-user-guide/01-client-interfaces/03-s3.md
@@ -1,3 +1,190 @@
+---
+sidebar_label: S3 API
+---
+
# S3 API
-**TODO:** File a subtask under
[HDDS-9858](https://issues.apache.org/jira/browse/HDDS-9858) and complete this
page or section.
+Ozone provides S3 compatible REST interface to use the object store data with
any S3 compatible tools.
+S3 buckets are stored under the `/s3v` volume.
+
+## Getting started
+
+S3 Gateway is a separated component which provides the S3 compatible APIs. It
should be started additional to the regular Ozone components.
+You can start a Docker based cluster, including the S3 Gateway from the
release package.
+
+Go to the `compose/ozone` directory, and start the server:
+
+```bash
+docker-compose up -d --scale datanode=3
+```
+
+You can access the S3 Gateway at `http://localhost:9878`
+
+## URL Schema
+
+Ozone S3 Gateway supports both the virtual-host-style URL S3 bucket addresses
(eg. http://bucketname.host:9878) and the path-style addresses (eg.
http://host:9878/bucketname)
+By default it uses the path-style addressing. To use virtual host style URLs
set your main domain name in your `ozone-site.xml`:
+
+```xml
+<property>
+ <name>ozone.s3g.domain.name</name>
+ <value>s3g.internal</value>
+</property>
+```
+
+## Implemented REST Endpoints
+
+The Ozone S3 Gateway implements a substantial subset of the Amazon S3 REST
API. The tables below summarize each API’s support status, its feature, and any
relevant notes, including known deviations from AWS S3 behavior.
+
+### Bucket Operations
+
+| **API Name** | **Feature** | **Note** |
+|
---------------------------------------------------------------------------------------------------
| ------------- | ---------- |
+| ✅
[ListBuckets](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListBuckets.html)
| Lists all buckets owned by the authenticated user. | Returns the full bucket
list. |
+| ✅
[CreateBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateBucket.html)
| Creates a new bucket. | **Non-compliant behavior:** The default bucket ACL
may include extra group permissions instead of being strictly private. Bucket
names must adhere to S3 naming conventions. |
+| ✅
[HeadBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadBucket.html)
| Checks for the existence of a bucket. | Returns a 200 status if the bucket
exists. |
+| ✅
[DeleteBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucket.html)
| Deletes a bucket. | Bucket must be empty before deletion. |
+| ✅
[GetBucketLocation](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketLocation.html)
| Retrieves the location (region) of a bucket. | Typically returns a default
region (e.g., `us-east-1`), which may differ from AWS if region-specific
responses are expected. |
+
+### Object Operations
+
+| **API Name** | **Feature** | **Note** |
+|
-------------------------------------------------------------------------------------------
| ------------- | ---------- |
+| ✅
[GetObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html)
| Retrieves the contents of an object. | **Non-compliant behavior:** For
non-existent objects, Ozone may return a generic 404 without the structured XML
error body defined by AWS S3. |
+| ✅
[HeadObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadObject.html)
| Retrieves metadata for an object. | Similar to GET Object, error responses
for missing objects might lack detailed AWS-style error XML. |
+| ✅
[PutObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html)
| Uploads a new object. | Supports both single PUT and multipart uploads.
**Non-compliant behavior:** Any ACL headers provided in the request are
ignored. |
+| ✅
[DeleteObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObject.html)
| Deletes a single object. | Returns standard error codes for missing objects.
|
+| ✅
[DeleteObjects](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjects.html)
| Deletes multiple objects in one request. | Bulk delete operation with
aggregated error reporting. |
+| ✅
[CopyObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html)
| Copies an object (intra- or inter-bucket). | Supports metadata directives
(COPY/REPLACE). **Non-compliant behavior:** Self-copying for metadata updates
may differ slightly from AWS S3. |
+| ✅
[ListObjectsV2](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html)
| Lists objects in a bucket with filtering and pagination. | Supports prefix,
delimiter, and continuation tokens. |
+
+### Object Tagging Operations
+
+| **API Name** | **Feature** | **Note** |
+| ------------ | ----------- | -------- |
+| ✅
[PutObjectTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectTagging.html)
| Adds or updates tags on an object. | Fully supported; updates replace any
existing tag set. |
+| ✅
[GetObjectTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectTagging.html)
| Retrieves the tag set of an object. | Returns the tag set as defined by the
S3 API. |
+| ✅
[DeleteObjectTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjectTagging.html)
| Removes the tag set from an object. | Clears all tags associated with the
object. |
+
+### Multipart Upload Operations
+
+| **API Name** | **Feature** | **Note** |
+| ------------ | ----------- | -------- |
+| ✅
[CreateMultipartUpload](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateMultipartUpload.html)
| Starts a multipart upload session. | Returns an UploadId for subsequent
parts. |
+| ✅
[UploadPart](https://docs.aws.amazon.com/AmazonS3/latest/API/API_UploadPart.html)
| Uploads an individual part for a multipart upload. | Part numbering and ETag
computation follow AWS semantics. |
+| ✅
[ListMultipartUploads](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListMultipartUploads.html)
| Lists in-progress multipart uploads in a bucket. | Up to 1000 multipart
uploads are returned in a batch by default. |
+| ✅
[ListParts](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListParts.html)
| Lists uploaded parts for an in-progress multipart upload. | Supports
pagination if many parts exist. |
+| ✅
[CompleteMultipartUpload](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CompleteMultipartUpload.html)
| Assembles all uploaded parts into a final object. | Returns an ETag that
follows AWS multipart rules. |
+| ✅
[AbortMultipartUpload](https://docs.aws.amazon.com/AmazonS3/latest/API/API_AbortMultipartUpload.html)
| Aborts an in-progress multipart upload. | Frees up storage used by uploaded
parts. Incomplete multipart uploads older than 7 days are automatically aborted
by Ozone. |
+
+### Additional Operations
+
+| **API Name** | **Feature** | **Note** |
+| -------------------------- | ------------- | ---------- |
+| ✅ Generate Presigned URL | Generates a temporary URL for accessing an
object. | Uses AWS Signature V4. **Non-compliant behavior:** The generated URL
may include a fixed default region rather than dynamically reflecting the
bucket’s location. Ozone now supports generating presigned URLs for all major
S3 operations, including `GetObject`, `PutObject`, `DeleteObject`,
`HeadObject`, `HeadBucket`, `MultipartUpload`. |
+
+:::note Additional Notes
+
+- **Advanced Features Not Supported:**
+ - **ACLs, Bucket Policies, CORS Configuration, and Website Hosting:** These
are not fully implemented; Ozone uses an internal permission model that
deviates from AWS S3.
+ - **Bucket Versioning, Object Locking, Server-Side Encryption, and S3
Select:** These features are currently not supported.
+ - **Conditional Requests:** Support for conditional requests (e.g.,
`If-Match`, `If-None-Match`) is planned and tracked in
[HDDS-13117](https://issues.apache.org/jira/browse/HDDS-13117).
+ - **Lifecycle configuration, cross region replication, S3 event
notifications** are being implemented and in the roadmap.
+- While Ozone S3 Gateway provides extensive support for common S3 operations,
users should be aware of the above non-compliant behaviors and limitations when
integrating with applications expecting full AWS S3 functionality.
+- While Ozone S3 Gateway does not support S3 Server-Side Encryption, it does
support encrypted buckets using Apache Ranger KMS. For more information, see
the [Transparent Data
Encryption](../../05-administrator-guide/02-configuration/03-security/05-encryption/02-transparent-data-encryption.md)
documentation.
+:::
+
+## Security
+
+If security is not enabled, you can *use* **any** AWS_ACCESS_KEY_ID and
AWS_SECRET_ACCESS_KEY
+
+If security is enabled, you can get the key and the secret with the `ozone s3
getsecret` command (*Kerberos based authentication is required).
+
+```bash
+kinit -kt /etc/security/keytabs/testuser.keytab testuser/[email protected]
+ozone s3 getsecret
+awsAccessKey=testuser/[email protected]
+awsSecret=c261b6ecabf7d37d5f9ded654b1c724adac9bd9f13e247a235e567e8296d2999
+```
+
+:::note
+Starting in Ozone 1.4.0, the secret will be **shown only once** when generated
with `getsecret`. If the secret is lost, the user would have to `revokesecret`
first before regenerating a new secret with `getsecret`.
+:::
+
+Now, you can use the key and the secret to access the S3 endpoint:
+
+```bash
+export AWS_ACCESS_KEY_ID=testuser/[email protected]
+export
AWS_SECRET_ACCESS_KEY=c261b6ecabf7d37d5f9ded654b1c724adac9bd9f13e247a235e567e8296d2999
+aws s3api --endpoint http://localhost:9878 create-bucket --bucket bucket1
+```
+
+To invalidate/revoke the secret, use `ozone s3 revokesecret` command.
Parameter '-y' can be appended to skip the interactive confirmation.
+
+```bash
+ozone s3 revokesecret
+Enter 'y' to confirm S3 secret revocation for 'testuser/[email protected]': y
+S3 secret revoked.
+```
+
+Ozone Manager administrators can run `ozone s3 getsecret` and `ozone s3
revokesecret` command with `-u` parameter to specify another users.
+
+```bash
+# Obtained Kerberos TGT for testuser/[email protected] with kinit,
+# testuser/[email protected] is an OM admin.
+ozone s3 getsecret -u om/[email protected]
+awsAccessKey=om/[email protected]
+awsSecret=1e9379d0424cce6669b1a501ff14834e46dee004ee868b41a313b49eabcfb68f
+
+ozone s3 revokesecret -u om/[email protected] -y
+S3 secret revoked.
+```
+
+## Expose any volume
+
+Ozone has one more element in the name-space hierarchy compared to S3: the
volumes. By default, all the buckets of the `/s3v` volume can be accessed with
S3 interface but only the (Ozone) buckets of the `/s3v` volumes are exposed.
+
+To make any other buckets available with the S3 interface a "symbolic linked"
bucket can be created:
+
+```bash
+ozone sh volume create /s3v
+ozone sh volume create /vol1
+
+ozone sh bucket create /vol1/bucket1
+ozone sh bucket link /vol1/bucket1 /s3v/common-bucket
+```
+
+This example expose the `/vol1/bucket1` Ozone bucket as an S3 compatible
`common-bucket` via the S3 interface.
+
+:::note
+The implementation details of the bucket-linking feature can be found in the
[design doc](https://ozone.apache.org/docs/edge/design/volume-management.html).
+:::
+
+## Clients
+
+### AWS CLI
+
+`aws` CLI could be used by specifying the custom REST endpoint.
+
+```bash
+aws s3api --endpoint http://localhost:9878 create-bucket --bucket buckettest
+```
+
+Or
+
+```bash
+aws s3 ls --endpoint http://localhost:9878 s3://buckettest
+```
+
+## Compatible third-party applications
+
+Ozone's S3 Gateway enables integration with a wide range of cloud-native and
analytics applications. Here are some examples of tools and platforms known to
work with Ozone (in alphabetical order):
+
+- [Clickhouse](https://clickhouse.com/docs/en/integrations/s3)
+- [Fluentd](https://docs.fluentd.org/output/s3) can send logs directly to
Ozone via the S3 Gateway.
+- [JuiceFS](https://juicefs.com/docs/community/s3_gateway/)
+- [Starburst](https://docs.starburst.io/latest/connector/starburst-ozone.html)
(Starburst also supports Ozone `ofs://`)
+- [Teradata
NOS](https://docs.teradata.com/r/Enterprise_IntelliFlex_VMware/Teradata-VantageTM-Native-Object-Store-Getting-Started-Guide-17.20)
+- [CyberDuck](https://cyberduck.io/) See the [tutorial
page](https://ozone.apache.org/docs/edge/interface/cyberduckozones3.html)
+
+This list is not exhaustive—any application that supports the S3 protocol can
potentially connect to Ozone, making it easy to adopt Ozone in modern data
pipelines and cloud-native workflows.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
