JingsongLi commented on PR #8139: URL: https://github.com/apache/paimon/pull/8139#issuecomment-4850739587
Thanks for digging into the DV case. I agree we should not turn a DV-only restore into DELETE + ADD if the streaming read path cannot carry and apply the snapshot-specific deletion vectors correctly. Making the restore non-empty but logically wrong for streaming is worse than keeping it metadata-only. So my preference is: 1. Keep this restore operation metadata-only. 2. For normal data-file differences, still write the overwrite delta from the previous latest snapshot to the target snapshot, so streaming overwrite readers can observe file-level restores. 3. For DV-only differences, do not synthesize DELETE + ADD unless the streaming diff path is made DV-aware and covered by an end-to-end changelog test. 4. Document the limitation explicitly: batch and time-travel reads are correct because the new snapshot points to the target index manifest, but DV-only restores are not guaranteed to produce a correct streaming overwrite changelog. One more thought about the procedure name: I am not fully convinced by `restore_as_latest`. The operation is logically closer to a non-destructive rollback: restore the table state to an older snapshot/tag, but keep later snapshots/tags and materialize the restored state as a new latest snapshot. Maybe a rollback-family name would make the relation to the existing `rollback_to` clearer. For example, `rollback_to_as_latest` is a bit long, but it makes both parts explicit: rollback to a target version, and make that restored state the latest snapshot without deleting later history. `rollback_to_latest` sounds shorter, but may be read as "rollback to the latest", so I think it is less clear. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
