Hi devs,
I'd like to propose dropping `renameBranch` from the REST Catalog API and
gather feedback before moving forward.
Problem
The REST Catalog exposes `renameBranch` (POST
/v1/.../branches/{branch}/rename). Server-side implementation
backed by object storage (OSS) has to implement this via the underlying store's
rename,
which is not atomic — typically copy + delete with its own visibility window.
During that window, concurrent writes to the source branch can be silently
lost: the writer sees the source branch
still exists and commits to it, but those commits land in a directory that gets
deleted (or was copied before the write)
by the rename. We don't have a clean fencing mechanism on the server side to
atomically block or redirect
writers for the duration of the rename, and we don't see a straightforward fix
today.
Proposal
Stop exposing `renameBranch` over the REST Catalog:
- Remove `RESTApi.renameBranch` and the `RenameBranchRequest` payload.
- `RESTCatalog.renameBranch` would throw `UnsupportedOperationException`.
- Drop the corresponding mock-server route and the rename assertions in
`RESTCatalogTest#testBranches`.
What is NOT being removed:
- The `Catalog#renameBranch` interface method.
- Non-REST implementations: file-system branch manager, Spark/Flink
`RenameBranchProcedure`, etc.
File-system backends like HDFS support atomic directory rename, so the
data-loss window is REST/object-store-specific.
Reference
Draft PR: https://github.com/apache/paimon/pull/8023
<https://github.com/apache/paimon/pull/8023 >
Particularly interested in input from anyone running REST Catalog in
production with branch workflows, or who sees a path to making this
safe that we've missed.
Thanks,
Kevin (Qingwei Yang / 轻为)
