SteNicholas opened a new pull request, #357: URL: https://github.com/apache/paimon-cpp/pull/357
### Purpose Linked issue: close #351 Java Paimon addresses a branch by the `<table>$branch_<branch>` object name and hands that branch down to `SnapshotCommit#commit(baseSnapshotUuid, snapshot, branch, statistics)`, while `AbstractFileStoreTable#switchToBranch` reads the branch's schema from `branch/branch-<name>` through a `FileSystemSchemaManager`. Paimon C++ had none of this on the commit path: `FileStoreCommit::Create()` built its `SchemaManager` and `SnapshotManager` without a branch, so a commit configured with `branch=dev` read the main schema and published its snapshot under the main branch's directory, and a write through a catalog refused a branch outright with `a write through a catalog requires the main branch`. This PR makes a branch a first-class target of the write and commit path, for a file-system commit, a catalog commit and a request-only commit alike. **Resolving the branch** - `BranchManager::ResolveBranch()` resolves the branch an operation is aimed at from every source that may name it: the table identifier, the `branch` option and, where a builder has one, `WithBranch()`. Names that disagree are refused rather than chosen between, and `tbl$branch_main`, `branch=main`, an empty and a blank name all mean the main branch. A read resolves it the same way, so the `branch` option now names a read's branch as `WithBranch()` already did. - Naming a branch only in the option while the identifier of `WithCatalog()` names the bare table is refused: the catalog would answer for the table while the files were written for the branch. **Addressing the branch on a catalog** - `Identifier(database, table, branch)` builds the `<table>$branch_<branch>` object name, folding every spelling of `main` into the bare table name as the Java client does. `CatalogUtils::BranchIdentifier()` rebuilds it for every catalog call, so what is read and what is committed name one object; `RestCatalog` now goes through the same builder instead of splicing the name itself. - `SnapshotCommit::Commit()` takes the branch. `CatalogSnapshotCommit` commits to the branch's object name, and `RenamingSnapshotCommit` writes the snapshot through a manager of that branch. - `UseRESTCatalogCommit()` still builds a body that names no branch, so the caller sends it to the branch's own URL: the commit endpoint of `tbl$branch_dev`, not the one of `tbl`. - Two kinds of name a catalog cannot tell apart from another object are refused where a builder sees them: a branch differing from `main` by case alone, whose snapshots would go to a directory of their own, and a branch holding a `$`, which reads back as a system table of another branch. **Schema of a branch** - A branch keeps its schema, snapshots and real-time offsets under `branch/branch-<name>`, while data and manifests stay shared under the table path. A write or a commit aimed at a branch reads the schema published there rather than the catalog's, as `switchToBranch` does, so a catalog serving only the table's own current schema still takes the branch's snapshots. Only the main branch reloads the catalog's schema id for a new snapshot. - This library creates no branch: one has to exist, with its schema published under `branch/branch-<name>`, before a write or a commit can be aimed at it. Catalog branch management (creating, deleting and merging a branch) remains unimplemented. **Expiration** `Expire()` and snapshot loading follow the commit's branch, so expiring on a branch manages that branch's snapshots and its `EARLIEST` hint. As in Java's `ExpireSnapshotsImpl`, only the retained snapshots of that branch are read while every branch shares the table's data files, so a file that only another branch still refers to is deleted. This is stated in `include/paimon/file_store_commit.h` and called out as a warning in the catalog guide; cross-branch reachability belongs to orphan-file cleaning, as it does upstream. ### Tests New and extended UT: - `identifier_test.cpp`: `BranchConstructor` (object name, round trip, and every spelling of `main` folding to the bare table); a blank branch added to `InvalidEmptySystemTableNameParts`. - `branch_manager_test.cpp`: `TestResolveBranch` (each source, agreement, disagreement, names that would leave the table root), `TestCheckCatalogAddressableBranch`. - `catalog_snapshot_commit_test.cpp`: `TestCommitToBranch` (branch object name per attempt, the branch of the attempt winning over the one the identifier was built for, `MAIN` refused); `TestBuildRequestWithoutCatalog` extended so a refused attempt leaves no request behind. - `renaming_snapshot_commit_test.cpp`: `TestCommitToBranch` (snapshot and `LATEST` under the branch, main untouched, and a commit aimed back at main from a branch manager). - `commit_context_test.cpp` / `write_context_test.cpp`: `TestCatalogAddressesBranchByIdentifier`, `TestBranch`. `read_context_test.cpp`: `TestBranch`. - `file_store_commit_impl_test.cpp`: `TestCommitToBranch`, `TestRequestOnlyCommitToBranch`, `TestCatalogCommitToBranchTakesTheBranchSnapshot`, `TestBranchCommitReadsTheSchemaFromTheBranch`, `TestBranchCommitNamesTheBranchWhenItFails`, `TestCatalogCommitCanonicalizesTheMainBranchIdentifier`, `TestBranchExpireLeavesMainSnapshotMetadataAlone`. - `file_store_write_test.cpp`: `TestCatalogWriteAsksTheCatalogOnlyForTheMainBranchSchema`. - `rest_catalog_test.cpp`: `ListSnapshots` extended with a blank branch, which is the bare table as `main` and `MAIN` are. New and extended IT: - `write_inte_test.cpp`: `TestBranchCommitAndRead` writes and commits on `dev` and on main, then reads each back. - `pk_compaction_inte_test.cpp`: `WriteAndCompactWithBranch` now commits the compaction to `branch-rt` and scans the branch for the compacted file, instead of verifying the commit messages only. ### API and Format Yes. - `include/paimon/catalog/identifier.h`: new `Identifier(database, table, branch)` constructor. - `include/paimon/file_store_commit.h`: `GetLastCommitTableRequest()` now clears the request of the previous attempt, so an attempt that failed before building one returns an error rather than the request before it. `Expire()` documents that only the retained snapshots of its own branch are read. - `include/paimon/commit_context.h`, `include/paimon/write_context.h`, `include/paimon/read_context.h`: `WithCatalog()`, `WithBranch()` and the `branch` option document how a branch is named and how disagreeing names are refused. A commit or a catalog write aimed at a branch is accepted where it was refused before. - No storage format change: `<table>$branch_<branch>` and `branch/branch-<name>` are the existing Java spellings. ### Documentation Yes. - `docs/source/user_guide/catalog.rst`: a new **Committing to a branch** section with **Naming the branch**, **What a branch keeps and what it shares** and **Addressing a branch without a catalog client**; the expiration warning covering data files shared by every branch; and a note that branch management covers creating, deleting and merging a branch while committing to an existing one is supported. - `docs/source/user_guide/write.rst`: the main branch loads its current schema from the catalog while a branch reads the schema published under `branch/branch-<name>`, and a request-only commit aimed at a branch has to be sent to that branch's URL. ### Generative AI tooling Generated-by: Claude Code (claude-opus-5[1m]) 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
