gmhelmold opened a new pull request, #2814:
URL: https://github.com/apache/iceberg-rust/pull/2814
## Which issue does this PR close?
- Part of #823.
First table slice of the metadata-tables epic, as announced on the epic
thread (refs / history / metadata_log_entries are the opening slices).
## What changes are included in this PR?
Adds the `refs` metadata table, exposing a table's snapshot references
(branches and tags) as rows, matching Iceberg's reference `RefsTable.java`.
Schema (field ids 1–6, identical to the reference `SNAPSHOT_REF_SCHEMA`):
| id | column | type | required |
|----|---------------------------|--------|----------|
| 1 | name | string | ✓ |
| 2 | type | string | ✓ |
| 3 | snapshot_id | long | ✓ |
| 4 | max_reference_age_in_ms | long | optional |
| 5 | min_snapshots_to_keep | int | optional |
| 6 | max_snapshot_age_in_ms | long | optional |
- `type` is the uppercase ref kind — `"BRANCH"` / `"TAG"` — matching Java's
`SnapshotRefType.name()`.
- Retention columns follow the reference mapping: a **branch** fills
`min_snapshots_to_keep`, `max_snapshot_age_in_ms`, and
`max_reference_age_in_ms`; a **tag** fills only `max_reference_age_in_ms`,
leaving the other two null.
- Rows are emitted name-ordered. `TableMetadata::refs` is a `HashMap`, so
the scan collects into a `BTreeMap` to keep output deterministic.
- No public getter is added; the scan reads the existing `pub(crate)` field
directly, so the public surface is unchanged.
Registry wiring: `MetadataTableType::Refs` (+ `as_str` / `TryFrom`), the
`MetadataTable::refs()` accessor, and the DataFusion `TableProvider`
schema/scan arms. The `$refs` DataFusion table name is enumerated automatically
via `MetadataTableType::all_types()` (strum `EnumIter`).
## Are these changes tested?
Yes.
- Two unit goldens in `inspect::refs::tests`:
- `test_refs_table` uses the shared `TableTestFixture` (a `main` branch
auto-injected by V2 normalization with all-null retention, plus a `test` tag).
It pins row order, the `BRANCH`/`TAG` type strings, and the tag's
`max_reference_age_in_ms`.
- `test_refs_table_retention_fields` uses inline metadata (TempDir-based,
deterministic) whose `main` branch carries **explicit** retention — the
load-bearing golden for the branch column mapping (`min_snapshots_to_keep=5`,
`max_snapshot_age_in_ms=86400000`, `max_reference_age_in_ms=259200000`), which
the all-null fixture branch does not exercise.
- DataFusion / sqllogictest goldens: `test_provider_list_table_names` now
lists `my_table$refs`; `show_tables.slt` lists the `$refs` tables;
`insert_into.slt` selects `$refs` on a snapshot-less table and expects **zero**
rows (V2 normalization only injects `main` once a current snapshot exists).
Red→green verified locally: flipping the branch type string `"BRANCH"` →
`"branch"` fails both unit goldens on the type column; restoring passes. `cargo
clippy --all-targets --all-features --workspace -- -D warnings` clean.
## Are there any user-facing changes?
New feature. `SELECT * FROM <table>$refs` (via DataFusion) and
`table.inspect().refs()` now expose a table's snapshot references.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
