leaves12138 commented on code in PR #9726: URL: https://github.com/apache/paimon/pull/9726#discussion_r3980320849
########## docs/docs/pypaimon/branches.md: ########## @@ -0,0 +1,180 @@ +--- +title: "Branches and Rollback" +description: "Use branches to maintain separate table histories and rollback to restore a retained snapshot." +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Branches and Rollback + +Use branches to maintain separate table histories and rollback to restore a retained snapshot. To pin a dataset for later reads or training, use [tags](./manage-tags). The examples operate on an existing `table`. + +## Branch Management + +A branch maintains a separate table history. Creating a branch without a tag +copies the current schema into an empty branch. To include existing data, create +the branch from a retained tag. + +:::info + +PyPaimon provides two implementations of `BranchManager`: +- **FileSystemBranchManager**: For tables accessed directly via filesystem (default for filesystem catalog) +- **CatalogBranchManager**: For tables accessed via catalog (e.g., REST catalog) + +The `table.branch_manager()` method automatically returns the appropriate implementation based on the table's catalog environment. + +::: + +### Create Branch + +Create an empty branch with the current table schema: + +```python +from pypaimon import CatalogFactory + +catalog = CatalogFactory.create({'warehouse': 'file:///path/to/warehouse'}) +table = catalog.get_table('database_name.table_name') + +# Create an empty branch with the current schema +table.branch_manager().create_branch('feature_branch') +``` + +Alternatively, create a branch that includes the data retained by an existing tag: + +```python +# Create a branch from tag 'v1.0' +table.branch_manager().create_branch('feature_branch', tag_name='v1.0') +``` + +Create a branch and ignore if it already exists: + +```python +# No error if branch already exists +table.branch_manager().create_branch('feature_branch', ignore_if_exists=True) +``` + +### List Branches + +List all branches for a table: + +```python +# Get all branch names +branches = table.branch_manager().branches() + +for branch in branches: + print(f"Branch: {branch}") +``` + +### Check Branch Exists + +Check if a specific branch exists: + +```python +if table.branch_manager().branch_exists('feature_branch'): + print("Branch exists") +else: + print("Branch does not exist") +``` + +### Drop Branch + +Delete an existing branch: + +```python +# Drop a branch +table.branch_manager().drop_branch('feature_branch') +``` + +### Rename Branch + +Rename an existing branch to a new name: + +```python +# Rename a branch +table.branch_manager().rename_branch('old_branch_name', 'new_branch_name') +``` + +:::warning + +The source branch must exist and cannot be the main branch. The target branch name must be valid and not already exist. + +::: + +### Fast Forward + +Fast forward the main branch to a specific branch: + +```python +# Fast forward main to feature branch +# This is useful when you want to merge changes from a feature branch back to main +table.branch_manager().fast_forward('feature_branch') +``` + +:::warning + +Fast forward operation is irreversible and will replace the current state of the main branch with the target branch's state. Review Comment: Non-blocking follow-up: this warning was already present in `python-api.mdx`, so this is not a regression introduced by the reorganization, but it does not hold for the current filesystem implementation. I reproduced the following with a local append table on this head: 1. Commit `{id: 1}` as snapshot 1 and append `{id: 2}` as snapshot 2. 2. Create tag `v1` on snapshot 1, then `create_branch("feature", tag_name="v1")`. 3. Call `table.branch_manager().fast_forward("feature")` and reopen the main table. The feature branch reads only `id=1`, but main still selects snapshot 2 and reads both rows after the call. `FileSystemBranchManager.fast_forward()` in `paimon-python/pypaimon/branch/filesystem_branch_manager.py` copies branch snapshot files but does not remove the later main snapshots (the implementation explicitly leaves that cleanup unimplemented). Could we qualify the filesystem behavior here, or track completion of that implementation separately, rather than promise that main's state is always replaced? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
