The GitHub Actions job "Check License" on fluss.git/spark-kv-read-bugfix has 
failed.
Run started by GitHub user beryllw (triggered by beryllw).

Head commit for run:
cd0aaf55898195998a4b494ee17537bae418d085 / Junbo Wang <[email protected]>
[client] Fix data hole in primary key table batch read

SortMergeReader merged the snapshot and change log driven solely by the
snapshot iterator, so once the snapshot was exhausted the trailing change
log records (keys greater than the max snapshot key) were silently dropped.
Spark's non-lake primary key batch read (FlussUpsertPartitionReader) calls
readBatch() only once and hit this directly, producing a data hole where
COUNT(*) < max_seq - min_seq + 1.

Rewrite SortMergeReader as a single two-way merge iterator so the merged
output is independent of how many times readBatch() is called (Spark calls
it once, the lake scanner calls it repeatedly). Add a unit test covering
change log keys beyond the max snapshot key and an end-to-end Spark ITCase.

Report URL: https://github.com/apache/fluss/actions/runs/29001909196

With regards,
GitHub Actions via GitBox

Reply via email to