markhoerth opened a new issue, #14422:
URL: https://github.com/apache/gravitino/issues/14422
### Version
main branch, commit bb634defdc5663a8124be5fca1c8d9e52ac096ac (2.0.0-SNAPSHOT)
### Describe what's wrong
A user creates a `system_iceberg_orphan_file_removal` policy whose
`location` has an inner space, tab, newline or other control character, for
example `s3://bucket/tbl/da ta`. The policy is accepted and stored (200). When
the optimizer builds the cleanup job, it parses the location with
`java.net.URI.create`, which throws `IllegalArgumentException: Illegal
character in path`. Each run for that policy then fails, long after the write
that caused it, instead of the user getting a 400 when the policy is written.
The cause is in
`api/src/main/java/org/apache/gravitino/policy/IcebergOrphanFileRemovalContent.java:146-170`.
`validate()` says it mirrors `OrphanFileLocationUtils.normalizeLocation`
(comment at `:159`) and rejects `..`, `\`, `%`, `?` and `#`, but it never
parses the value as a URI. At run time, `GravitinoOrphanFileRemovalJobAdapter`
(`maintenance/optimizer/src/main/java/org/apache/gravitino/maintenance/optimizer/recommender/job/GravitinoOrphanFileRemovalJobAdapter.java:82-83`)
and `IcebergRemoveOrphanFilesJob`
(`maintenance/jobs/src/main/java/org/apache/gravitino/maintenance/jobs/iceberg/IcebergRemoveOrphanFilesJob.java:117`,
`:198`) call `OrphanFileLocationUtils.normalizeLocation`
(`maintenance/optimizer-api/src/main/java/org/apache/gravitino/maintenance/optimizer/common/util/OrphanFileLocationUtils.java:59-66`),
which calls `URI.create(value)` at `:66`.
### Error message and/or stacktrace
Write time:
```
POST /api/metalakes/ml/policies content
{"olderThanDays":3,"location":"s3://bucket/tbl/da ta"}
200
{"code":0,"policy":{"name":"orph_sp",...,"content":{"olderThanDays":3,"location":"s3://bucket/tbl/da
ta","dryRun":false},...}}
```
What `URI.create` throws on that value at run time:
```
java.lang.IllegalArgumentException: Illegal character in path at index 18:
s3://bucket/tbl/da ta
```
### How to reproduce
1. Create a metalake `ml`:
```
curl -u admin:x -H 'Content-Type: application/json' -H 'Accept:
application/vnd.gravitino.v1+json' -X POST http://127.0.0.1:8090/api/metalakes
-d '{"name":"ml","comment":"","properties":{}}'
```
2. Create the policy; it is accepted:
```
curl -u admin:x -H 'Content-Type: application/json' -H 'Accept:
application/vnd.gravitino.v1+json' -X POST
http://127.0.0.1:8090/api/metalakes/ml/policies -d
'{"name":"orph_sp","comment":"","policyType":"system_iceberg_orphan_file_removal","enabled":true,"content":{"olderThanDays":3,"location":"s3://bucket/tbl/da
ta"}}'
```
3. For comparison, a `?` in the location is refused with 400 "location must
not contain query or fragment".
4. Run `java.net.URI.create("s3://bucket/tbl/da ta")` (what
`normalizeLocation` does) to see the run-time failure.
### Additional context
Related: #14348 (write-time validation of other orphan removal job options,
closed). The 2.0 OSS test suite check
`test_orphan_location_that_is_not_a_uri_rejected` found it. The defect is the
gap between Gravitino's write-time and run-time validation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
