LuciferYang opened a new issue, #11029:
URL: https://github.com/apache/paimon/issues/11029

   ### Search before asking
   
   - [X] I searched in the [issues](https://github.com/apache/paimon/issues) 
and found nothing similar.
   
   ### Paimon version
   
   master (b000c3945f)
   
   ### Compute Engine
   
   Python
   
   ### Minimal reproduce step
   
   Create an append table with `'bucket' = '1'` and a `bucket-key`. Commit six 
one-row batches with PyPaimon (`seq` = 0..5), then write one more row (`seq` = 
6) with a Java writer at default settings and read the table.
   
   ### What doesn't meet your expectations?
   
   The rows come back as `[2, 0, 1, 5, 4, 3, 6]` instead of `[0, 1, 2, 3, 4, 5, 
6]`. The docs say a bucketed append table keeps append order within a partition 
and bucket.
   
   PyPaimon writes every file of a bucketed append table with 
`min_sequence_number = max_sequence_number =` the bucket's restored max, so all 
six files have the range `(1, 1)`. Java's `AppendOnlyWriter` numbers rows from 
the restored max + 1. Six files reach `compaction.min.file-num` (5 by default), 
so the Java write compacts the bucket. `BucketedAppendCompactManager` picks 
files from a queue ordered by `minSequenceNumber`, and with equal numbers that 
queue does not return them in append order.
   
   ### Anything else?
   
   PyPaimon alone does not compact append tables, so the order is only lost 
once a Java writer compacts the bucket.
   
   ### Are you willing to submit a PR?
   
   - [X] I'm willing to submit a PR!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to