RyanLin5967 opened a new issue, #10887: URL: https://github.com/apache/paimon/issues/10887
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version pypaimon 2.1.0. `fast_forward` is unchanged on master. ### Compute Engine Python API (pypaimon 2.1.0, Python 3.12.16, pyarrow 19.0.1) on a local filesystem catalog. Spark 3.5.6 with paimon-spark-3.5_2.12 2.1.0 for the comparison. ### Minimal reproduce step Currently, pypaimon's `fast_forward` copies the branch's snapshot files into main without first deleting main's snapshots from the branch point on, so once main has more snapshots than the branch, main keeps reading its own. Spark's `CALL sys.fast_forward` on the same table gives `[1, 2]`. ```python import pyarrow as pa from pypaimon import CatalogFactory, Schema cat = CatalogFactory.create({"warehouse": "wh"}) cat.create_database("db", True) sch = pa.schema([("k", pa.int64())]) cat.create_table("db.t", Schema.from_pyarrow_schema(sch), False) def write(name, k): wb = cat.get_table(name).new_batch_write_builder() w, c = wb.new_write(), wb.new_commit() w.write_arrow(pa.table({"k": [k]}, schema=sch)) c.commit(w.prepare_commit()) def read(name): rb = cat.get_table(name).new_read_builder() return sorted(rb.new_read().to_arrow(rb.new_scan().plan().splits())["k"].to_pylist()) write("db.t", 1) cat.get_table("db.t").create_tag("v1", 1) cat.create_branch("db.t", "b", "v1") write("db.t$branch_b", 2) write("db.t", 3) write("db.t", 4) cat.fast_forward("db.t", "b") print(read("db.t")) # [1, 3, 4], should be [1, 2] print(read("db.t$branch_b")) # [1, 2] ``` ### What doesn't meet your expectations? After `fast_forward("db.t", "b")` main holds the branch's state and reads `[1, 2]`. ### Anything else? If the branch has added a column, reading main after `fast_forward` raises `AttributeError: 'NoneType' object has no attribute 'fields'`, since the branch's schema files are not copied either. ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
