leaves12138 opened a new pull request, #9392: URL: https://github.com/apache/paimon/pull/9392
### Purpose Flink may push a projection into a Paimon lookup source. Paimon appends primary-key and partition fields to that projection for internal lookup and dynamic-partition handling, but currently returns those internal fields to Flink as well. In async lookup joins, Flink then copies a row whose arity is larger than the planned serializer arity. This closes #6878. ### Approach - Keep the augmented projection for the internal lookup table. - Record the fields originally requested by Flink. - Project matched rows back to the requested fields before returning them from `FileStoreLookupFunction`. - Avoid the extra wrapper when the internal and output projections are identical. ### Tests - Added a direct lookup test for an internally appended primary-key field. - Added an async lookup join test for dynamic partition projection, covering both primary-key and append-only tables. - Verified the new tests with Flink 1.20 and Flink 2.2. - Verified existing max-partition, async-retry, BLOB lookup, and all `FileStoreLookupFunctionTest` cases with Flink 2.2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
