markhoerth opened a new issue, #14428:
URL: https://github.com/apache/gravitino/issues/14428
### Version
main branch, commit bb634defdc5663a8124be5fca1c8d9e52ac096ac (2.0.0-SNAPSHOT)
### Describe what's wrong
A user sends `POST /api/metalakes/{metalake}/policies` with a `policyType`
that is not a known built-in type: a typo such as `system_iceberg_bogus`, or a
built-in type in the wrong case such as `SYSTEM_ICEBERG_COMPACTION`. The user
expects a 400 that says the policy type is unknown. Instead the request is
rejected with a 400 whose reason is `supportedObjectTypes cannot be empty`, a
rule for custom policy content that has nothing to do with the mistake.
The message changes with the content sent, and is never about the type
unless the content happens to be valid custom content:
- Empty content: `supportedObjectTypes cannot be empty`.
- Compaction content such as `{"maxPartitionNum":5}` with
`SYSTEM_ICEBERG_COMPACTION`: `Malformed json request`.
- Custom content with `SYSTEM_ICEBERG_COMPACTION`: `Expected
IcebergDataCompactionContent for policy type ICEBERG_COMPACTION, but got
org.apache.gravitino.policy.PolicyContents$CustomContent`.
- Custom content with `system_iceberg_bogus`: only now the intended `Unknown
built-in policy type: system_iceberg_bogus`.
A mis-cased built-in type can therefore never be created, but it is also
never reported as a wrong type.
Cause (Gravitino code):
-
`common/src/main/java/org/apache/gravitino/dto/requests/PolicyCreateRequest.java:52-57`:
`content` is mapped with `@JsonTypeInfo(... property = "policyType",
defaultImpl = PolicyContentDTO.CustomContentDTO.class)`. Jackson matches
subtype names case-sensitively, so any `policyType` that is not an exact
registered name, including a built-in name in another case, is deserialized as
custom content.
-
`server/src/main/java/org/apache/gravitino/server/web/rest/PolicyOperations.java:164-165`:
`request.validate()` runs before
`validateCreatePolicyType(request.getPolicyType())`. `validate()` calls
`policyContent.validate()` (`PolicyCreateRequest.java:122`), which applies the
custom-content rule at
`api/src/main/java/org/apache/gravitino/policy/PolicyContent.java:65-66` before
the type is checked.
- `api/src/main/java/org/apache/gravitino/policy/Policy.java:86-89`
(`BuiltInType.fromPolicyType`) and `PolicyOperations.java:400-409`
(`validateCreatePolicyType`) compare with `equalsIgnoreCase`, so they treat
`SYSTEM_ICEBERG_COMPACTION` as the compaction type, while the content was
already deserialized as custom content. Creation then fails later with the
type-mismatch message.
### Error message and/or stacktrace
```
POST /api/metalakes/vrf_pol_ml/policies
{"policyType":"system_iceberg_bogus","content":{}, ...}
HTTP 400
{"code":1001,"type":"IllegalArgumentException","message":"Failed to operate
policy(s) [p_bogus] operation [CREATE] under object [vrf_pol_ml], reason
[supportedObjectTypes cannot be empty]"}
java.lang.IllegalArgumentException: supportedObjectTypes cannot be empty
at
org.apache.gravitino.policy.PolicyContent.validate(PolicyContent.java:65)
at
org.apache.gravitino.dto.requests.PolicyCreateRequest.validate(PolicyCreateRequest.java:122)
at
org.apache.gravitino.server.web.rest.PolicyOperations.lambda$createPolicy$5(PolicyOperations.java:164)
POST ... {"policyType":"SYSTEM_ICEBERG_COMPACTION","content":{}}
HTTP 400 ... reason [supportedObjectTypes cannot be empty]
POST ...
{"policyType":"SYSTEM_ICEBERG_COMPACTION","content":{"maxPartitionNum":5}}
HTTP 400
{"code":1001,"type":"IllegalArgumentException","message":"Malformed json
request"}
POST ...
{"policyType":"SYSTEM_ICEBERG_COMPACTION","content":{"customRules":{"a":1},"supportedObjectTypes":["TABLE"],"properties":{}}}
HTTP 400 ... reason [Expected IcebergDataCompactionContent for policy type
ICEBERG_COMPACTION, but got
org.apache.gravitino.policy.PolicyContents$CustomContent]
POST ...
{"policyType":"system_iceberg_bogus","content":{"customRules":{"a":1},"supportedObjectTypes":["TABLE"],"properties":{}}}
HTTP 400 ... reason [Unknown built-in policy type: system_iceberg_bogus]
```
### How to reproduce
1. Create a clean metalake:
```
curl -s -X POST -H 'Authorization: Basic YWRtaW46eA==' -H 'Content-Type:
application/json' \
http://localhost:8090/api/metalakes -d
'{"name":"vrf_pol_ml","comment":"","properties":{}}'
```
2. Create a policy with an unknown built-in type and empty content. The
response is a 400 with `supportedObjectTypes cannot be empty`:
```
curl -s -X POST -H 'Authorization: Basic YWRtaW46eA==' -H 'Content-Type:
application/json' \
http://localhost:8090/api/metalakes/vrf_pol_ml/policies \
-d
'{"name":"p_bogus","comment":"","policyType":"system_iceberg_bogus","enabled":true,"content":{}}'
```
3. Create a policy with a built-in type in upper case. The response is the
same 400:
```
curl -s -X POST -H 'Authorization: Basic YWRtaW46eA==' -H 'Content-Type:
application/json' \
http://localhost:8090/api/metalakes/vrf_pol_ml/policies \
-d
'{"name":"p_upper","comment":"","policyType":"SYSTEM_ICEBERG_COMPACTION","enabled":true,"content":{}}'
```
4. Repeat step 3 with `"content":{"maxPartitionNum":5}`. The response is a
400 with `Malformed json request`.
5. Repeat step 3 with
`"content":{"customRules":{"a":1},"supportedObjectTypes":["TABLE"],"properties":{}}`.
The response is a 400 with `Expected IcebergDataCompactionContent for policy
type ICEBERG_COMPACTION, but got ...CustomContent`.
6. Repeat step 2 with the content from step 5. Only now the response is
`Unknown built-in policy type: system_iceberg_bogus`.
### Additional context
Related: #8673 (custom policy type restriction). The cause is Gravitino's
own request handling (the JSON default subtype and the order of validation in
`PolicyOperations.createPolicy`), not Iceberg.
Found by the 2.0 OSS test suite checks
`test_unknown_or_miscased_policy_type_named_in_error[system_iceberg_bogus]` and
`test_unknown_or_miscased_policy_type_named_in_error[SYSTEM_ICEBERG_COMPACTION]`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
