zhangyue19921010 opened a new issue, #11188:
URL: https://github.com/apache/paimon/issues/11188

   ### Search before asking
   
   - [x] I searched in the issues and found nothing similar.
   
   ### Paimon version
   
   2.2-SNAPSHOT, commit `f1ffd1369ae6db2438a4c9a98fe4216566e88ccc`.
   
   ### Compute Engine
   
   Java API, JDK 11.0.18.
   
   ### Minimal reproduce step
   
   Create an ordinary append table with one `id INT` column and these options:
   
   ```text
   bucket = 1
   bucket-key = id
   bucket-append-ordered = true
   target-file-row-num = 1
   compaction.min.file-num = 3
   async-file-write = false
   ```
   
   Using one `StreamWriteBuilder`, write IDs 0 through 4 and commit 
`write.prepareCommit(true, 0)`. Automatic compaction merges the first three 
files; a fresh initial streaming scan reads `[0, 1, 2, 3, 4]` from the three 
remaining files.
   
   Then call `write.compact(BinaryRow.EMPTY_ROW, 0, true)` and commit 
`write.prepareCommit(true, 1)` with commit identifier 1. One file remains, but 
the same scan now reads `[0, 1, 2, 4, 3]`.
   
   ### What doesn't meet your expectations?
   
   With `bucket-append-ordered = true`, full compaction should preserve append 
order within the same partition and bucket.
   
   Automatic compaction requeues its small output before the two later files. 
The queue is a priority queue, but its iterator is not sorted. 
`FullCompactTask` copies that iteration order into the rewrite input, so the 
final file stores later rows out of order. Sorting the candidate list by its 
minimum sequence number before rewriting preserves the contract.
   
   ### Anything else?
   
   This reproduction uses only Java writes and distinct sequence ranges. The 
earlier Python sequence-number issue #11029 has a different cause.
   
   Prepared with TRAE CLI.
   
   ### Are you willing to submit a PR?
   
   - [x] I'm willing to submit a PR!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to