zhangyue19921010 opened a new issue, #11206:
URL: https://github.com/apache/paimon/issues/11206

   ### Search before asking
   
   - [x] I searched in the issues and found nothing similar.
   
   ### Paimon version
   
   2.2-SNAPSHOT, commit `00b4d05a6cc118091a38eeca23549519cfb775f4`.
   
   ### Compute Engine
   
   Java batch API, JDK 11.0.18.
   
   ### Minimal reproduce step
   
   Create an ordinary append table `(a INT, b INT)` with `bucket=1`, 
`bucket-key=a`, `bucket-append-ordered=false`, `clustering.incremental=true`, 
`clustering.columns=a` and `num-levels=3`.
   
   1. A batch writer writes `(1,3),(2,2),(3,1)`, performs full compaction and 
prepares its messages.
   2. Change `clustering.columns` to `b` through the catalog. A writer using 
the updated table writes `(1,6),(2,5),(3,4)`, performs full compaction and 
prepares its messages.
   3. Gather both writers' messages in one batch commit, placing the current 
writer first. The snapshot contains two real files at level 2, with different 
producing schema IDs. The six `b` values read correctly.
   4. Run full compaction through a new writer using the current table. It 
selects no work and leaves both files, including the one clustered by `a`. 
Reversing the prepared-message order lets full compaction rewrite them into one 
file clustered by `b`.
   
   ### What doesn't meet your expectations?
   
   After changing clustering columns, full compaction should apply them across 
the selected scope. A restored higher-level run can contain files from 
different producing schemas. `IncrementalClusterStrategy` checks only its first 
file's schema before skipping, so a file using the current columns can hide 
files using older columns.
   
   Check the clustering columns of every distinct schema in the run. Files with 
different schema IDs but the same clustering columns should retain the existing 
skip behavior.
   
   ### Anything else?
   
   The reproduction uses actual Java writers, catalog schema changes, 
prepared-message collection, commit and read APIs. No data-file metadata is 
fabricated. A same-key control changes only file compression across schemas and 
verifies that full compaction keeps those files.
   
   AI-generated with TRAE CLI (GPT-6.1-Sol).
   
   ### Are you willing to submit a PR?
   
   - [x] I'm willing to submit a PR!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to