JingsongLi commented on PR #8508:
URL: https://github.com/apache/paimon/pull/8508#issuecomment-4921462728
`compact_chain_table(..., partial partition, overwrite=true)` may cause data
loss. Using the partial `partitionSpec` in
`sinkBuilder.overwrite(partitionSpec)` will delete all groups under the
specified partial partition in the snapshot branch. However, the source side's
`skipPreloadTargetSnapshot` only generates output from target chain partitions
present in the delta. If a snapshot group already exists but has no
corresponding delta in the target chain partition, it will be overwritten and
deleted without being rewritten. The author is advised to add a scenario:
partitioned by `(region, dt, hr)`, chain keys `(dt, hr)`, with a snapshot
containing `region=FR, dt=..., hr=...`, and a delta only including `region=CN,
dt=..., hr=...`. After executing `compact_chain_table('dt=..., hr=...', true)`,
the `FR` data should not be lost.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
