xndai opened a new pull request, #1124:
URL: https://github.com/apache/iceberg-cpp/pull/1124

   Closes #1123.
   
   ## What
   
   Adds three helpers to `LocationUtil` 
(`src/iceberg/util/location_util.{h,cc}`) for v4 relative paths, as defined in 
the spec's [Paths in 
Metadata](https://iceberg.apache.org/spec/#paths-in-metadata) section 
(apache/iceberg#15630):
   
   - `HasScheme(location)`: true when the text before the first `:` is a 
non-empty RFC 3986 scheme. This is the spec's test for an absolute path. It is 
strict on every platform: unlike `ParseScheme`, a single letter (`C:`) counts 
as a scheme.
   - `ResolveLocation(table_location, location) -> std::string`: absolute paths 
are returned unchanged; relative paths become `table_location + "/" + 
location`, with no `/` added when the table location already ends in one. 
`.`/`..` are not interpreted, and a leading `/` on the path is kept (`//`), as 
the spec's example shows.
   - `RelativizeLocation(table_location, location) -> std::string_view`: when 
`location` starts with `table_location` immediately followed by `/`, returns 
the rest after that separator; otherwise returns `location` unchanged. A table 
location that ends in `/` already supplies the separator.
   
   Nothing calls them yet. Using them in metadata, manifests and catalogs is 
left to follow-up PRs.
   
   ## Trailing separator on the table location
   
   The spec recommends that a table location not end in `/`, because the join 
would otherwise produce `//` (its "Duplicate separator" example). This PR 
applies that recommendation more strongly: when the table location ends in `/`, 
`ResolveLocation` doesn't add a second separator (`s3://bucket/db/table/` + 
`data/00000-0.parquet` → `s3://bucket/db/table/data/00000-0.parquet`). Object 
stores such as S3 treat `//` as part of the key and advise against it. 
`RelativizeLocation` accepts such a location the same way, so relativize and 
resolve still round-trip. A leading `/` on the relative path is kept: 
relativize produces one only when the stored path really contains `//`.
   
   ## Comparison with Java
   
   Matches Java's `LocationUtil.hasScheme` / `resolveLocation` / 
`relativizeLocation` (apache/iceberg#16174, apache/iceberg#17807), with these 
differences:
   - Java always adds the separator and doesn't relativize against a table 
location ending in `/`; this PR handles the trailing `/` as described above.
   - `RelativizeLocation` returns a `std::string_view` into its `location` 
argument, because the result is always a suffix of that argument and copying it 
isn't needed.
   
   Edge cases that match Java: only one separator is stripped, so `<loc>//x` 
relativizes to `/x` and resolves back to `<loc>//x`, and the table location 
itself is kept absolute.
   
   ## Not covered: adding a scheme to pre-v4 paths
   
   The spec's [Path 
Resolution](https://iceberg.apache.org/spec/#path-resolution) section ends with 
a table of examples. Its last row, "Missing scheme", is about v3 and earlier 
tables: a stored path without a scheme, such as 
`/wh/db/table/data/00000-0.parquet`, should get one added 
(`hdfs://wh/db/table/data/00000-0.parquet` in the example). That path is 
already complete, so this isn't relative resolution, and the spec doesn't say 
where the scheme comes from. This PR doesn't do it; that is follow-up work. For 
that row, the test only checks that such a path isn't treated as absolute 
(`HasScheme` is false). The other rows are tested through `ResolveLocation`.
   
   ## Tests
   
   `src/iceberg/test/location_util_test.cc`:
   - Every row of the spec's path resolution table.
   - `HasScheme` grammar cases: leading digit or `+`, `_`, `/` before the 
colon, empty input, the Windows extended-length form, drive letters.
   - No normalization of `.`/`..`, plus both kinds of duplicate separator.
   - Sibling prefixes (`/t` vs `/t2`), the table location itself, different and 
case-differing schemes, trailing slashes.
   - Relativize-then-resolve round trips, with and without a trailing `/` on 
the table location.
   
   `icb build -j 12`, the full `icb test -j 12` (21/21) and `icb precommit` all 
pass.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to