yiyutian1 opened a new pull request, #29980:
URL: https://github.com/apache/flink/pull/29980

   ## What is the purpose of the change
   
   In batch mode, a query calling an `AsyncTableFunction` through `LATERAL` 
fails with "Cannot generate a valid execution plan for the given query: 
Unsupported function for scan:ASYNC_TABLE". This PR plans it as an async 
correlate on `AsyncWaitOperator`, sharing the planner logic with the streaming 
async correlate.
   
   This is the async table function counterpart of #29939 (FLINK-41482, async 
scalar functions in batch). The two PRs are independent and can be merged in 
either order.
   
   ## Notes to reviewers
   
   The PR has two commits: the first contains the actual logic and docs, the 
second only tests.
   
   ## Brief change log
   
     - Add `BatchPhysicalAsyncCorrelateRule`, `BatchPhysicalAsyncCorrelate` and 
exec node `BatchExecAsyncCorrelate` (`batch-exec-async-correlate`, version 1), 
reusing `CommonExecAsyncCorrelate`. `BatchPhysicalCorrelateRule` skips async 
table functions, and `BatchPhysicalConstantTableFunctionScanRule` handles them 
like its streaming counterpart.
     - Extract `CommonPhysicalAsyncCorrelateRule` from 
`StreamPhysicalAsyncCorrelateRule`, and move the projection and condition check 
into `AsyncTableUtil#validateAsyncCorrelate`, so stream and batch share them. 
The streaming behaviour doesn't change.
     - Add `AsyncCorrelateSplitRule.CORRELATE_SPLIT_ASYNC_TABLE` and 
`FlinkLogicalCalcRemoveRule` to the batch logical rewrite, matching the 
streaming rule set.
     - Keep `BatchExecAsyncCorrelate` out of multiple-input operators, since 
`TableOperatorWrapper` doesn't set the mailbox executor that 
`AsyncWaitOperatorFactory` needs. Let it propagate distribution in 
`ForwardHashExchangeProcessor` like `BatchExecCorrelate`.
     - Mark the `table.exec.async-table.*` options as batch and streaming, and 
document that a retry still pending when a bounded input ends gets one more 
attempt.
   
   ## Verifying this change
   
   This change added tests and can be verified as follows:
   
     - `AsyncCorrelateTest` (batch plan tests), including left join, scalar 
arguments, a constant async table function, the unsupported projection and 
condition cases, and multiple-input handling.
     - `AsyncCorrelateBatchRestoreTest`, using `AsyncCorrelateTestPrograms`, 
which moves to `exec/common`.
     - `AsyncCorrelateBatchSemanticTests`: calls still in flight when the input 
ends, a scalar function in the async table function's argument, and an async 
correlate between an aggregation and a join (this one fails without the 
multiple-input exclusion).
     - `ForwardHashExchangeTest` case for an async correlate between a hash 
join and a hash aggregate.
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): no
     - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: no (only the documented exec mode of existing options)
     - The serializers: no
     - The runtime per-record code paths (performance sensitive): no
     - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
     - The S3 file system connector: no
   
   ## Documentation
   
     - Does this pull request introduce a new feature? yes
     - If yes, how is the feature documented? docs
   
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   - [X] Yes (please specify the tool below)
   
   Generated-by: Claude Code (Claude Opus 5.5)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to