nsivarajan opened a new pull request, #61329:
URL: https://github.com/apache/doris/pull/61329
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
### Problem Summary:
Doris cloud mode has no native Alibaba Cloud OSS storage vault support.
Users on Alibaba Cloud ECS cannot use ECS instance profile credentials or
STS AssumeRole when accessing OSS, forcing them to manage static AK/SK keys.
This PR adds a native OSS storage vault type (`type = "OSS"`) with support
for ECS instance profile, STS AssumeRole, and explicit AK/SK authentication.
This PR introduces a native OSS storage vault type (`type = "OSS"`) that:
- Uses the official Alibaba Cloud OSS C++ SDK for all object operations
- Supports ECS instance profile credentials (no AK/SK required when a RAM
role
is attached to the ECS instance)
- Supports STS AssumeRole (`oss.role_arn`) for cross-account access and
least-privilege access patterns
- Supports explicit AK/SK for environments without instance profiles
- Integrates with the cloud recycler for object lifecycle management
- Wires into the existing storage vault creation, alteration, and show
commands
**ECS instance profile (no credentials required):**
```sql
CREATE STORAGE VAULT oss_vault
PROPERTIES (
"type" = "OSS",
"oss.endpoint" = "oss-cn-beijing-internal.aliyuncs.com",
"oss.region" = "cn-beijing",
"oss.bucket" = "my-bucket",
"oss.root.path" = "doris/data"
);
STS AssumeRole:
CREATE STORAGE VAULT oss_vault_assumerole
PROPERTIES (
"type" = "OSS",
"oss.endpoint" = "oss-cn-beijing-internal.aliyuncs.com",
"oss.region" = "cn-beijing",
"oss.bucket" = "my-bucket",
"oss.root.path" = "doris/data",
"oss.role_arn" = "acs:ram::123456789:role/doris-role"
);
Explicit AK/SK:
CREATE STORAGE VAULT oss_vault_aksk
PROPERTIES (
"type" = "OSS",
"oss.endpoint" = "oss-cn-beijing-internal.aliyuncs.com",
"oss.region" = "cn-beijing",
"oss.bucket" = "my-bucket",
"oss.root.path" = "doris/data",
"oss.access_key" = "<ak>",
"oss.secret_key" = "<sk>"
);
CREATE STORAGE VAULT oss_vault_assumerole
PROPERTIES (
"type" = "OSS",
"oss.endpoint" = "oss-cn-beijing-internal.aliyuncs.com",
"oss.region" = "cn-beijing",
"oss.bucket" = "my-bucket",
"oss.root.path" = "doris/data",
"oss.role_arn" = "acs:ram::123456789:role/doris-role",
"oss.external_id" = "my-external-id"
);
```
### Release note
Add native Alibaba Cloud OSS storage vault support (`type = "OSS"`) with
ECS
instance profile, STS AssumeRole, and explicit AK/SK authentication modes.
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [X] Regression test
- [X] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- Behavior changed:
- [ ] No.
- [X] Yes.
- New `OSS` storage vault type added alongside existing `S3` and
`HDFS`
types. Existing S3 and HDFS vault behavior is unchanged.
- New property keys introduced: `oss.endpoint`, `oss.bucket`,
`oss.region`, `oss.root.path`, `oss.access_key`, `oss.secret_key`,
`oss.role_arn`, `oss.external_id`
- BE build now links against Alibaba Cloud OSS SDK and STS SDK v2
when
`USE_OSS=ON` (default)
- Does this need documentation?
- [X] No.
- [ ] Yes. <!-- Document PR to be added covering CREATE STORAGE VAULT
syntax
for OSS type, supported authentication modes, and
property
reference -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
