markhoerth opened a new issue, #14437:
URL: https://github.com/apache/gravitino/issues/14437
### Version
main branch, commit bb634defdc5663a8124be5fca1c8d9e52ac096ac (2.0.0-SNAPSHOT)
### Describe what's wrong
Every read path shows a secret-backed property as the placeholder `******`.
Alter refuses that placeholder as a new secret value: a `setSecretBinding`
change with `"plaintext":"******"` is rejected with 400 "Property 'api-token'
cannot be set to the masked placeholder value '******'". Create does not apply
the same check. A catalog (or schema) created with
`"secretBindings":{"api-token":{"provider":"memory","plaintext":"******"}}` is
accepted with 200, and the literal string `******` is written to the secret
provider as the secret value. The user expects the same 400 as on alter. It
shows up when a user copies an object's definition from a GET response (where
the secret is already masked) and creates a new object from it: the create
succeeds, the new object reports `"api-token":"******"` like any healthy
secret, and the connector later receives `******` as the credential.
Cause (defect in Gravitino's own code):
- Alter:
`core/src/main/java/org/apache/gravitino/catalog/CatalogManager.java:1715-1720`
puts the `setSecretBinding` plaintext into the upserts map, and
`CatalogManager.java:1031-1036` passes it to `validatePropertyForAlter`, which
calls `HiddenPropertyMaskUtils.validateNoMaskedPlaceholders`
(`core/src/main/java/org/apache/gravitino/catalog/PropertiesMetadataHelpers.java:93`,
check at
`core/src/main/java/org/apache/gravitino/connector/HiddenPropertyMaskUtils.java:93-106`).
- Create: `CatalogManager.java:832-839` merges the bindings into the catalog
configuration as secret URNs and calls `SecretManager.assembleSecretMaterials`.
`core/src/main/java/org/apache/gravitino/secret/SecretManager.java:139-158` and
`toSecretMaterials` (`SecretManager.java:546-556`) turn each
`binding.plaintext()` into secret material with no placeholder check, and
`checkSecretKeys` (`SecretManager.java:81`) only checks keys and raw URNs. The
create-time `validatePropertyForCreate` (`PropertiesMetadataHelpers.java:52`)
runs on the property map, where the binding value has already been replaced by
its URN, so it never sees the plaintext.
- Schema and fileset create share the same path through
`assembleSecretMaterials`
(`core/src/main/java/org/apache/gravitino/catalog/SchemaOperationDispatcher.java:160`,
`core/src/main/java/org/apache/gravitino/catalog/FilesetOperationDispatcher.java:166`).
Schema create was confirmed to accept `******` as well.
Validating every `SecretBinding.plaintext()` with
`HiddenPropertyMaskUtils.isMaskedPlaceholder` inside `assembleSecretMaterials`
would cover catalog, schema and fileset create at once.
### Error message and/or stacktrace
Create with the placeholder is accepted:
```
POST /api/metalakes/kms_ml/catalogs -> 200
{"code":0,"catalog":{"name":"c_mask","type":"fileset","provider":"fileset","comment":"","properties":{"location":"file:///tmp/kms_c_mask","api-token":"******","in-use":"true"},"audit":{"creator":"admin",...}}}
```
The same value on alter is refused:
```
PUT /api/metalakes/kms_ml/catalogs/c_ok -> 400
{"code":1001,"type":"IllegalArgumentException","message":"Failed to operate
catalog(s) [c_ok] operation [ALTER] under metalake [kms_ml], reason [Property
'api-token' cannot be set to the masked placeholder value
'******']","stack":["java.lang.IllegalArgumentException: Property 'api-token'
cannot be set to the masked placeholder value '******'","\tat
org.apache.gravitino.connector.HiddenPropertyMaskUtils.validateNoMaskedPlaceholders(...)",
...]}
```
### How to reproduce
Server with a secret provider configured, for example:
```
gravitino.secret.providers = memory
gravitino.secret.provider.memory.className =
org.apache.gravitino.secret.memory.InMemorySecretsProvider
```
```
A="Authorization: Basic $(printf 'admin:x' | base64)"
G=http://127.0.0.1:8090/api; J='Content-Type: application/json'
```
1. Create a clean metalake and check the provider:
```
curl -s -H "$A" -H "$J" -X POST $G/metalakes -d
'{"name":"kms_ml","comment":"","properties":{}}'
curl -s -H "$A" $G/metalakes/kms_ml/secrets/providers
```
2. Create a catalog whose secret binding plaintext is the placeholder
(returns 200):
```
curl -s -w '\n%{http_code}\n' -H "$A" -H "$J" -X POST
$G/metalakes/kms_ml/catalogs -d
'{"name":"c_mask","type":"FILESET","provider":"fileset","comment":"","properties":{"location":"file:///tmp/kms_c_mask"},"secretBindings":{"api-token":{"provider":"memory","plaintext":"******"}}}'
```
3. Create a catalog with a real secret, then try to set the placeholder
through alter (returns 400):
```
curl -s -H "$A" -H "$J" -X POST $G/metalakes/kms_ml/catalogs -d
'{"name":"c_ok","type":"FILESET","provider":"fileset","comment":"","properties":{"location":"file:///tmp/kms_c_ok"},"secretBindings":{"api-token":{"provider":"memory","plaintext":"real-value-1"}}}'
curl -s -w '\n%{http_code}\n' -H "$A" -H "$J" -X PUT
$G/metalakes/kms_ml/catalogs/c_ok -d
'{"updates":[{"@type":"setSecretBinding","property":"api-token","provider":"memory","plaintext":"******"}]}'
```
4. Schema create behaves like step 2 (returns 200):
```
curl -s -w '\n%{http_code}\n' -H "$A" -H "$J" -X POST
$G/metalakes/kms_ml/catalogs/c_ok/schemas -d
'{"name":"s_mask","comment":"","properties":{},"secretBindings":{"api-token":{"provider":"memory","plaintext":"******"}}}'
```
### Additional context
- Related: #12581 (masked placeholder for hidden credential properties),
#12381 (setSecretBinding and setSecretReference on alter), #14396 (KMS-backed
secret encryption epic).
- Found by the 2.0 OSS test suite check
`test_mask_as_plaintext_rejected_on_create`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
