steFaiz opened a new pull request, #8964:
URL: https://github.com/apache/paimon/pull/8964
### Purpose
Currently, we do not detect concurrent MERGE INTO UPDATE and MATERIALIZE DV
Compaction, which may cause data broken:
For exmaple:
Current Data range:
```
file1
column A, B
Range [0, 100]
```
here comes concurrent Merge Into, update column A
```
file2
column A
Range [0, 100]
```
If we have concurrent materialize DV compaction:
```
remove file 1
add new file:
file 3
column A, B
Range [101, 150]
```
If compaction is committed after merge into, the result would be:
```
file2 file3
column A column A, B
Range [0, 100] Range[101, 150]
```
This is wrong. We should prevent it. So I check each new snapshots, make
sure there is no concurrent update to deleted row ranges.
### Tests
See unit tests and ITcases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
