This is an automated email from the ASF dual-hosted git repository.
voonhous pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/master by this push:
new 3a3a32fb48f4 docs(hudi-notebooks): note S3A works with any
S3-compatible object store (#19423)
3a3a32fb48f4 is described below
commit 3a3a32fb48f476be80735cbea5b5eb3dd71b6e01
Author: Gonzalo Peña-Castellanos <[email protected]>
AuthorDate: Thu Aug 27 23:13:47 2026 -0500
docs(hudi-notebooks): note S3A works with any S3-compatible object store
(#19423)
* docs: note S3A works with any S3-compatible object store
Signed-off-by: Gonzalo Peña-Castellanos <[email protected]>
* docs(hudi-notebooks): make the S3A snippet valid XML and scope path-style
access
Angle-bracket placeholders inside <value> do not parse if the block is
pasted into
core-site.xml, so use plain placeholder text. The path-style property is
what MinIO and
most other providers need but Amazon S3 does not, so say so next to it the
way the
endpoint property already does.
---------
Signed-off-by: Gonzalo Peña-Castellanos <[email protected]>
Co-authored-by: voon <[email protected]>
---
hudi-notebooks/README.md | 36 ++++++++++++++++++++++++++++++++++++
1 file changed, 36 insertions(+)
diff --git a/hudi-notebooks/README.md b/hudi-notebooks/README.md
index ddabc6121ffc..479a55676d0b 100644
--- a/hudi-notebooks/README.md
+++ b/hudi-notebooks/README.md
@@ -79,6 +79,42 @@ This project provides a ready-to-use Docker Compose
environment for running Apac
- Spark, Hive, and Hudi configs are in `conf/` and automatically copied into
containers.
- S3 access keys and endpoints are set for MinIO and referenced in Spark/Hive
configs.
+### Using another S3-compatible object store
+
+Hudi reads and writes through the Hadoop S3A connector, so the same `fs.s3a.*`
+settings work against any S3-compatible object store: Amazon S3, or a
compatible
+provider such as Backblaze B2, Cloudflare R2, or MinIO. This demo points
+`fs.s3a.endpoint` at the bundled MinIO service; to target a different store,
+edit `conf/hadoop/core-site.xml` (and the matching Spark/Hive configs) with
that
+provider's endpoint and credentials:
+
+```xml
+<property>
+ <name>fs.s3a.endpoint</name>
+ <!-- Amazon S3: https://s3.<region>.amazonaws.com
+ any other provider: its S3 endpoint URL -->
+ <value>https://your-s3-endpoint.example.com</value>
+</property>
+<property>
+ <name>fs.s3a.access.key</name>
+ <value>ACCESS_KEY</value>
+</property>
+<property>
+ <name>fs.s3a.secret.key</name>
+ <value>SECRET_KEY</value>
+</property>
+<property>
+ <name>fs.s3a.path.style.access</name>
+ <!-- MinIO and most other providers need path-style addressing;
+ leave this out for Amazon S3 -->
+ <value>true</value>
+</property>
+```
+
+For Amazon S3 the region-default endpoint is used when `fs.s3a.endpoint` is
+omitted; most other S3-compatible providers need an explicit endpoint and
+`fs.s3a.path.style.access=true`.
+
## 📒 Example: Using JupyterLab
1. Open [http://localhost:8888](http://localhost:8888) in your browser.
