lxy-9602 opened a new issue, #581:
URL: https://github.com/apache/paimon-cpp/issues/581

   ### Search before asking
   
   - [x] I searched in the 
[issues](https://github.com/apache/paimon-cpp/issues) and found nothing similar.
   
   
   ### Motivation
   
   External engines may build indexes for data files that already exist in a 
Paimon table. Paimon C++ needs an API to attach these serialized index payloads 
without rewriting the underlying data.
   
   This should support adding new field/type pairs and replacing existing 
indexes while preserving unrelated indexes and historical snapshots.
   
   ### Solution
   
   Introduce `FileIndexWrite::PrepareCommit`, accepting the table path, branch, 
options, optional base snapshot ID, per-file index payloads, and optional 
filesystem, memory pool, and executor instances.
   
   The API should:
   - Resolve live target files against a pinned snapshot.
   - Merge payloads into one external index container per data file.
   - Return existing commit message types grouped by partition and bucket for 
callers to commit.
   - Preserve reused data and auxiliary files when aborting metadata-only 
updates.
   
   Initially support fixed-bucket append tables and reject unsupported table 
modes.
   
   ### Anything else?
   
   Field names refer to the original data-file schema. Payloads use physical 
row positions, including rows marked deleted by deletion vectors.
   
   Backfill commits must be serialized with compaction/overwrite per partition 
and bucket. Callers are responsible for committing the returned messages and 
handling commit recovery.
   
   ### Are you willing to submit a PR?
   
   - [x] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to