xndai opened a new pull request, #1124:
URL: https://github.com/apache/iceberg-cpp/pull/1124
Closes #1123.
## What
Adds three helpers to `LocationUtil`
(`src/iceberg/util/location_util.{h,cc}`) for v4 relative paths, as defined in
the spec's [Paths in
Metadata](https://iceberg.apache.org/spec/#paths-in-metadata) section
(apache/iceberg#15630):
- `HasScheme(location)`: true when the text before the first `:` is a
non-empty RFC 3986 scheme. This is the spec's test for an absolute path. It is
strict on every platform: unlike `ParseScheme`, a single letter (`C:`) counts
as a scheme.
- `ResolveLocation(table_location, location) -> std::string`: absolute paths
are returned unchanged; relative paths become `table_location + "/" +
location`, with no `/` added when the table location already ends in one.
`.`/`..` are not interpreted, and a leading `/` on the path is kept (`//`), as
the spec's example shows.
- `RelativizeLocation(table_location, location) -> std::string_view`: when
`location` starts with `table_location` immediately followed by `/`, returns
the rest after that separator; otherwise returns `location` unchanged. A table
location that ends in `/` already supplies the separator.
Nothing calls them yet. Using them in metadata, manifests and catalogs is
left to follow-up PRs.
## Trailing separator on the table location
The spec recommends that a table location not end in `/`, because the join
would otherwise produce `//` (its "Duplicate separator" example). This PR
applies that recommendation more strongly: when the table location ends in `/`,
`ResolveLocation` doesn't add a second separator (`s3://bucket/db/table/` +
`data/00000-0.parquet` → `s3://bucket/db/table/data/00000-0.parquet`). Object
stores such as S3 treat `//` as part of the key and advise against it.
`RelativizeLocation` accepts such a location the same way, so relativize and
resolve still round-trip. A leading `/` on the relative path is kept:
relativize produces one only when the stored path really contains `//`.
## Comparison with Java
Matches Java's `LocationUtil.hasScheme` / `resolveLocation` /
`relativizeLocation` (apache/iceberg#16174, apache/iceberg#17807), with these
differences:
- Java always adds the separator and doesn't relativize against a table
location ending in `/`; this PR handles the trailing `/` as described above.
- `RelativizeLocation` returns a `std::string_view` into its `location`
argument, because the result is always a suffix of that argument and copying it
isn't needed.
Edge cases that match Java: only one separator is stripped, so `<loc>//x`
relativizes to `/x` and resolves back to `<loc>//x`, and the table location
itself is kept absolute.
## Not covered: adding a scheme to pre-v4 paths
The spec's [Path
Resolution](https://iceberg.apache.org/spec/#path-resolution) section ends with
a table of examples. Its last row, "Missing scheme", is about v3 and earlier
tables: a stored path without a scheme, such as
`/wh/db/table/data/00000-0.parquet`, should get one added
(`hdfs://wh/db/table/data/00000-0.parquet` in the example). That path is
already complete, so this isn't relative resolution, and the spec doesn't say
where the scheme comes from. This PR doesn't do it; that is follow-up work. For
that row, the test only checks that such a path isn't treated as absolute
(`HasScheme` is false). The other rows are tested through `ResolveLocation`.
## Tests
`src/iceberg/test/location_util_test.cc`:
- Every row of the spec's path resolution table.
- `HasScheme` grammar cases: leading digit or `+`, `_`, `/` before the
colon, empty input, the Windows extended-length form, drive letters.
- No normalization of `.`/`..`, plus both kinds of duplicate separator.
- Sibling prefixes (`/t` vs `/t2`), the table location itself, different and
case-differing schemes, trailing slashes.
- Relativize-then-resolve round trips, with and without a trailing `/` on
the table location.
`icb build -j 12`, the full `icb test -j 12` (21/21) and `icb precommit` all
pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]