Stephen0421 opened a new pull request, #10813:
URL: https://github.com/apache/paimon/pull/10813
### Purpose
`LocalTableQuery` builds lookup files without applying deletion vectors, and
checks only the row kind of the cached record. After compaction drops the
delete record and marks the old position with a deletion vector, Flink partial
lookup and the query service return the deleted row.
Deletion-vector tables that use the deduplicate merge engine and have no
sequence field now look up from level 0, so an uncompacted delete is visible.
The cached entry stores the row position. If that position is contained in the
bucket's current deletion vector, lookup returns null and does not continue
into older levels. The vector is not applied while building the lookup file,
because dropping the key there would fall through to a stale value.
Other merge engines and tables with a sequence field keep their previous
start level. A level-0 record is not yet merged, so returning it would skip
partial-update, aggregation, and sequence-field behavior. Those tables still
honor deletion vectors on the files they do read.
Each `refreshFiles` reloads the bucket deletion-vector index from the latest
snapshot. Streaming plans do not carry deletion files, and compaction can mark
a file that is absent from the current split. The parsed vectors are reused
while the index files are unchanged, and the cached file list is recorded only
after the read succeeds. Deletion-vector tables do not use remote lookup files,
because those files do not store the row position.
### Tests
- `LocalTableQueryDeletionVectorTest`
- An uncompacted level-0 delete is not returned, and a later update of
that key is returned.
- After compaction that only writes a deletion vector, the deleted key
stays invisible while another key in the same file remains visible.
- A partial-update level-0 record does not replace untouched columns with
null.
- An older sequence-field value does not overwrite the compacted value.
- `PrimaryKeySimpleTableTest#testTableQueryForNormal` and
`#testTableQueryForLookup` for the existing non-deletion-vector lookup path.
- `mvn -pl paimon-core spotless:check`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
