ump45nose opened a new pull request, #1342:
URL: https://github.com/apache/paimon-rust/pull/1342

   ### Purpose
   
   Linked issue: close #1341
   
   For tables managed by a REST catalog, the catalog owns tag and snapshot 
metadata. Several tag procedures still wrote the table's `tag/` directory 
directly, so the catalog and storage drifted apart without an error. 
`rollback_to_timestamp` also picked its target snapshot from storage listings.
   
   ### Brief change log
   
   Implements the solution proposed in the issue, in 
`crates/integrations/datafusion/src/procedures.rs`:
   
   - `create_tag`: on REST tables, calls `Catalog::create_tag_with_retention` 
with the requested `snapshot_id` (or `None` for latest) and `time_retained`. 
The catalog resolves the snapshot. `TagAlreadyExist` maps to the existing `Tag 
'...' already exists` error. Storage is not read or written.
   - `delete_tag`: on REST tables, calls `Catalog::delete_tag(.., 
ignore_if_not_exists = true)` for each comma-separated name. Missing tags are 
still skipped, as before.
   - `rename_tag`: refuses REST tables with `NotImplemented`, because there is 
no REST rename-tag endpoint. This matches `rename_branch`.
   - `create_tag_from_timestamp` and `rollback_to_timestamp`: refuse REST 
tables with `NotImplemented`, like `create_tag_from_watermark`. The check runs 
before any storage listing. For `rollback_to_timestamp`, this removes the REST 
`rollback_to` call that used a storage-selected snapshot. `rollback_to` with 
`snapshot_id`/`tag` still goes through the REST API.
   - `docs/src/sql.md`: documents the REST behavior of these procedures and 
adds a short `rename_tag` section, which was missing.
   
   The procedures use `rest_env.identifier()` for catalog calls, as 
`rollback_to` already does.
   
   ### Tests
   
   New tests in 
`crates/integrations/datafusion/tests/rest_management_procedures.rs`. Each 
seeds an unreadable storage file `tag/tag-release` and asserts it stays 
unchanged:
   
   - `test_create_and_delete_tag_go_through_rest_catalog`: covers `create_tag` 
with and without `snapshot_id` and `time_retained`, checking the tag through 
`RESTCatalog::get_tag`. A duplicate name returns `Tag 'release' already 
exists`. An unknown snapshot returns `Snapshot 2 does not exist` and creates no 
tag. `delete_tag` with `'release, missing,pinned'` removes both catalog tags 
and skips the missing one.
   - `test_storage_only_tag_procedures_rejected_on_rest_catalog`: `rename_tag`, 
`create_tag_from_timestamp` and `rollback_to_timestamp` return the REST 
`NotImplemented` errors.
   
   Without the fix in `procedures.rs`, both new tests fail.
   
   Ran locally:
   - `cargo test -p paimon-datafusion --test rest_management_procedures`: 26 
passed
   - `cargo test -p paimon-datafusion --test procedures`: 72 passed (I had to 
set `RUST_MIN_STACK` because the debug build overflowed the default test thread 
stack in my environment)
   - `cargo test -p paimon-datafusion --lib procedures`: 15 passed
   - `cargo fmt --all -- --check`
   
   Not run: the full workspace test suite and the CI clippy invocation. My 
local toolchain (1.99) reports new `must_use`-style clippy lints in untouched 
files (`catalog.rs`, `partition_row_count.rs`, `memory.rs`, `range_reader.rs`). 
The changed files produce no clippy diagnostics.
   
   ### API and Format
   
   No public API or storage format change. On REST tables, 
`create_tag`/`delete_tag` now change catalog state instead of storage files. 
`rename_tag`, `create_tag_from_timestamp` and `rollback_to_timestamp` now 
return `NotImplemented` instead of silently changing storage or choosing a 
storage-selected snapshot.
   
   ### Documentation
   
   Updated `docs/src/sql.md`.
   
   ---
   
   AI assistance disclosure: I prepared this change with help from an AI coding 
assistant. I reviewed the design and the code. One assumption: I follow the 
issue in calling the catalog's tag APIs for `create_tag`/`delete_tag` on REST 
tables, and I assume the `Catalog` passed to the procedure is the same REST 
catalog that loaded the table (it is in `execute_call`). Reviewers may also 
want `create_tag_from_timestamp`/`rollback_to_timestamp` to pass the timestamp 
to the server instead of refusing. I left that out because the Rust REST 
client's `Instant` only supports snapshot IDs and tags today.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to