wangxianghu opened a new pull request, #4360: URL: https://github.com/apache/amoro/pull/4360
## Why are the changes needed?
Close #4359.
AMS keeps sensitive values such as `admin-password` and
`database.password` in
`config.yaml`. Today the only shade providers are `base64` and the
built-in
default, so operators still have to store the real secret (obfuscated at
best)
on disk. When
AMS runs on AWS, the natural place for these secrets is AWS Secrets
Manager, with credentials resolved from the environment (EKS IRSA, ECS/EC2
instance profile, etc.) rather than checked into a file.
This PR adds an `aws-sm` `ConfigShade` implementation so that a sensitive
config
value can be an AWS Secrets Manager ARN, and AMS resolves the real value
at
startup:
```yaml
ams:
shade:
identifier: aws-sm
sensitive-keywords: database.username;database.password
database:
# both point to the same JSON secret; '#<field>' selects the value out
of it
username:
arn:aws:secretsmanager:ap-northeast-1:123456789012:secret:prod-amoro-db-mOhyOp#db.username
password:
arn:aws:secretsmanager:ap-northeast-1:123456789012:secret:prod-amoro-db-mOhyOp#db.password
```
- The region is resolved from the ARN — no extra config.
- A trailing #<field> selects one field when the secret value is JSON;
omit it when the secret value is the plaintext itself.
- Failures (malformed ARN, fetch failure, missing field) fail fast so AMS
never
starts with a wrong secret.
## Brief change log
- Add AwsSecretsManagerConfigShade (identifier = aws-sm): resolves a
secret
ARN into its value, extracting a JSON field when #<field> is given.
- Add SecretReference: parses
arn:aws:secretsmanager:<region>:<account>:secret:<name>[#<field>],
extracting the region from the ARN.
- Add AwsSecretsManagerClient + DefaultAwsSecretsManagerClient: a thin,
testable wrapper over the AWS SDK using the URL-connection HTTP client
(no
Netty), the default credentials provider chain, and per-process caching.
- A single client is created lazily and memoized; all shaded secrets are
expected to be in one region, and a cross-region reference fails fast.
- Register AwsSecretsManagerConfigShade in the ConfigShade SPI file.
- Add software.amazon.awssdk:secretsmanager (Netty/apache-client excluded)
and
url-connection-client dependencies.
## How was this patch tested?
- [x] Add some test cases that check the changes thoroughly including
negative and positive cases if possible
Added TestSecretReference (ARN/region/#field parsing, whitespace, and
every malformed-input rejection) and TestAwsSecretsManagerConfigShade
(plaintext vs JSON-field decryption, dotted field names taken literally
rather
than as a nested path, client memoization, cross-region fail-fast, and the
invalid-JSON / missing-field / malformed-ARN failure paths) — 23 tests,
all
passing.
- [ ] Add screenshots for manual tests if appropriate
- [x] Run test locally before making a pull request
## Documentation
- Does this pull request introduce a new feature? (yes )
- If yes, how is the feature documented? (JavaDocs)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
