skyyyy955 opened a new pull request, #10980:
URL: https://github.com/apache/paimon/pull/10980

   ### Purpose
   
   Fixes #10887. In RyanLin5967's example, `fast_forward` leaves main returning 
`[1, 3, 4]` instead of the branch's `[1, 2]`. Schema changes on the branch are 
also missing from main.
   
   This copies branch schemas, snapshots and tags into main, removes superseded 
main metadata and updates the snapshot hints while preserving earlier history. 
Retrying an interrupted copy does not require loading the partially copied 
metadata first. The replacement behavior follows the [merged Java 
implementation](https://github.com/apache/paimon/blob/30d91278af7435c95a1adf0062a44e8c3a05643d/paimon-core/src/main/java/org/apache/paimon/utils/FileSystemBranchManager.java).
   
   Affected local cache paths are invalidated before modifying metadata, 
including cached lengths and disk blocks reopened by a new catalog. This 
changes the internal disk-cache key layout: existing entries will need to be 
fetched again. Ordinary reads do not gain an extra remote metadata lookup.
   
   Opening this as a draft while the [cache-scope 
question](https://github.com/apache/paimon/issues/10887#issuecomment-6606405491)
 and issue assignment are pending. The patch does not make `fast_forward` 
atomic or refresh other running readers. An interrupted operation can leave 
main unreadable until a successful retry.
   
   ### Tests
   
   Validated against master `de6ac4a23094230047052a0fc42f595bfe1c99da`:
   
   - Linux, Python 3.10: 129 tests passed across 
`filesystem_catalog_branch_test`, `branch_manager_test`, 
`branch.file_store_table_branch_manager_test` and `caching_file_io_test`.
   - Windows, Python 3.10: 92 selected tests passed. Actual cached LocalFileIO 
scenarios require `os.pread` and were covered on Linux.
   - Six additional API scenarios passed: snapshot and schema ID collisions 
with caching disabled, memory caching and disk caching; 24 read comparisons 
matched the branch, including a new catalog reopening the disk cache.
   - Flake8 for the five changed files, Python license-header checks and `git 
diff --check` passed.
   
   Regression coverage includes reads and writes after fast-forward, retained 
history, interrupted-copy retries, cache invalidation failures and legacy 
disk-cache keys. Local API checks used ordinary append tables, the Python 
backend and local filesystems; concurrent readers, remote filesystems and 
Java/native interoperability have not been validated.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to