This is an automated email from the ASF dual-hosted git repository.
gaojun2048 pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/seatunnel.git
The following commit(s) were added to refs/heads/dev by this push:
new 94afa892b5 [Improve][Zeta] add document for Minio as checkpoint
storage for zeta tasks (#6732)
94afa892b5 is described below
commit 94afa892b5d29cb79097810ddc35f9eb7a952ccf
Author: gitfortian <[email protected]>
AuthorDate: Fri Apr 19 13:46:01 2024 +0800
[Improve][Zeta] add document for Minio as checkpoint storage for zeta tasks
(#6732)
---
docs/en/seatunnel-engine/checkpoint-storage.md | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
diff --git a/docs/en/seatunnel-engine/checkpoint-storage.md
b/docs/en/seatunnel-engine/checkpoint-storage.md
index c16084b02a..1e9a9332a5 100644
--- a/docs/en/seatunnel-engine/checkpoint-storage.md
+++ b/docs/en/seatunnel-engine/checkpoint-storage.md
@@ -124,6 +124,28 @@ seatunnel:
fs.s3a.aws.credentials.provider:
org.apache.hadoop.fs.s3a.InstanceProfileCredentialsProvider
```
+If you want to use Minio that supports the S3 protocol as checkpoint storage,
you should configure it this way:
+
+```yaml
+
+seatunnel:
+ engine:
+ checkpoint:
+ interval: 10000
+ timeout: 60000
+ storage:
+ type: hdfs
+ max-retained: 3
+ plugin-config:
+ storage.type: s3
+ fs.s3a.access.key: xxxxxxxxx # Access Key of MinIO
+ fs.s3a.secret.key: xxxxxxxxxxxxxxxxxxxxx # Secret Key of MinIO
+ fs.s3a.endpoint: http://127.0.0.1:9000 # Minio HTTP service access
address
+ s3.bucket: s3a://test # test is the bucket name which storage the
checkpoint file
+ fs.s3a.aws.credentials.provider:
org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
+ # important: The user of this key needs to have write permission for
the bucket, otherwise an exception of 403 will be returned
+```
+
For additional reading on the Hadoop Credential Provider API see: [Credential
Provider
API](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/CredentialProviderAPI.html).
#### HDFS