LuciferYang opened a new pull request, #11030: URL: https://github.com/apache/paimon/pull/11030
### Purpose close #11029 PyPaimon gave every file of a bucketed append table the bucket's restored max sequence number as its whole range. Java's compaction picks a bucket's files in sequence order, so once a Java writer compacts the bucket the rows lose their append order. The issue has the repro. Files of bucketed append tables now get the range of their own rows, starting at the bucket's max sequence number + 1, like Java's `AppendOnlyWriter`. Files of unaware-bucket tables keep the zero range; on row-tracking tables, which require `bucket = -1`, commit replaces it with the snapshot id. Primary-key tables are unchanged. ### Tests - `append_sequence_test.py`: three commits to a bucketed table get the ranges (2, 2), (3, 4) and (5, 5); on master all three are (1, 1). On unaware-bucket row-tracking and data-evolution tables the files keep (0, 0) and `_SEQUENCE_NUMBER` reads the snapshot id; this one also passes on master. - `test_py_write_bucketed_append_sequence` with `JavaPyE2ETest#testReadPythonBucketedAppendOrder`, added to `run_mixed_tests.sh`: PyPaimon commits six rows to one bucket, a Java write compacts the bucket, and Java reads the rows in append order. On master it reads `[2, 0, 1, 5, 4, 3, 6]`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
