JingsongLi opened a new issue, #1070:
URL: https://github.com/apache/paimon-rust/issues/1070

   ### Search before asking
   
   - [x] I searched in the issues and found nothing similar.
   
   ### Motivation
   
   PyPaimon already accepts a Paimon table as the source of batch and stream 
`merge_into`. Its native prepared MERGE path currently falls back to Python for 
that input, requiring Python to materialize the full source and perform the 
join.
   
   The prepared Rust core API only accepts Arrow batches or self-merge on 
`_ROW_ID`. The SQL MERGE path publishes its own commit, so it cannot replace 
the prepared API used by PyPaimon.
   
   ### Solution
   
   Allow a Paimon `Table` source in core `MergeSource`. Resolve the source's 
branch and snapshot selectors once, project keys and referenced columns, and 
use the ordinary logical reader, including primary-key merge engines. Keep 
source batches separate and gather matching rows across contributing chunks 
instead of concatenating the entire source.
   
   Expose that input through the existing thin Python binding. Preserve 
prepared-message ownership and Java/Paimon Spark MERGE semantics for NULL keys, 
cardinality checks and ordered clauses.
   
   ### Anything else?
   
   This uses existing Paimon Parquet/Blob readers and adds no storage format or 
dependency. Incremental source scans and unsupported catalog-backed Blob view 
resolution can continue on the existing Python path.
   
   ### Willingness to contribute
   
   - [x] I'm willing to submit a PR!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to