juntaozhang opened a new pull request, #8529:
URL: https://github.com/apache/paimon/pull/8529
### Purpose
When compact_chain_table is called with overwrite = true on a chain table
such as:
```
CREATE TABLE chain_t (
`t1` BIGINT COMMENT 't1',
`t2` BIGINT COMMENT 't2',
`t3` STRING COMMENT 't3'
) PARTITIONED BY (`region` STRING COMMENT 'region', `dt` STRING COMMENT
'dt', `hour` STRING COMMENT 'hour')
TBLPROPERTIES (
'dynamic-partition-overwrite' = 'true',
'chain-table.enabled' = 'true',
'primary-key' = 'region,dt,hour,t1',
'sequence.field' = 't2',
'bucket-key' = 't1',
'bucket' = '1',
'partition.timestamp-pattern' = '$dt $hour:00:00',
'partition.timestamp-formatter' = 'yyyyMMdd HH:mm:ss',
'merge-engine' = 'deduplicate',
'chain-table.chain-partition-keys' = 'dt,hour'
)
```
Users typically compact a chain partition by specifying only the chain
dimension:
>CALL sys.compact_chain_table('default.chain_t', 'dt=20250810,hr=22',
true);
Assume the snapshot branch already contains data for:
```
region='FR', dt='20250810', hr='22'
region='CN', dt='20250810', hr='22'
```
But the delta branch only contains new data for:
> region='CN', dt='20250810', hr='22'
If the table is configured with `'dynamic-partition-overwrite' = 'false'`,
the overwrite commit deletes all full partitions matching `dt=20250810, hr=22`
in the snapshot branch, including region='FR' which has no corresponding delta
data. As a result, the existing snapshot data for `region='FR'` is lost.
### Tests
- Extended CompactChainTableProcedureTest with a group-partition test that
sets `'dynamic-partition-overwrite' = 'false'` and verifies the snapshot-only
group is preserved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
