[
https://issues.apache.org/jira/browse/FLINK-41482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hao Li reassigned FLINK-41482:
------------------------------
Assignee: Hao Li
> Support AsyncScalarFunction in batch mode
> -----------------------------------------
>
> Key: FLINK-41482
> URL: https://issues.apache.org/jira/browse/FLINK-41482
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
> Affects Versions: 2.5.0
> Reporter: Yiyu Tian
> Assignee: Hao Li
> Priority: Major
> Labels: pull-request-available
>
> Async functions are only supported in streaming. FLIP-400
> (AsyncScalarFunction) and FLIP-498 (AsyncTableFunction) scoped batch out.
> In batch mode today, an AsyncScalarFunction call is planned by
> BatchPhysicalCalcRule as a regular BatchPhysicalCalc and fails in calc
> codegen with "Only scalar functions can be used in a projection or filter
> operation". The async table function counterpart is FLINK-37927.
> Proposal: mirror the streaming path in the batch planner.
> * The async calc split rules already run in sync UDFs).
> AddAsyncCorrelateSplitRule.CORRELATE_SPLIT_ASYNC_SCALAR so async calls in
> correlate arguments are split out too.
> * Add BatchPhysicalAsyncCalcRule / BatchPhyse BatchExecAsyncCalc,
> reusingCommonExecAsyncCalc, the async codegen and AsyncWaitOperator.
> * Keep the existing table.exec.async-scalar.* options (max concurrent
> operations, timeout, retry). Output mode stays ORDERED.
> * Batch input ends with DRAIN, so in-flight calls complete before the task
> finishes.
> * Keep async exec nodes out of batch multiple-input operators, since the
> multiple-input operator doesn't set the
> mailbox executor that AsyncWaitOperatorFacto
> This only turns a query that fails today into one that runs. Existing batch
> and streaming plans don't change.
> Known limitation: at end of input, AsyncWaitg retry one immediate attempt and
> thendisables retries. In batch, a call that fails more than once fails the
> job regardless of max-attempts. This PR documents it, and fixing it needs an
> operator change, which can be a follow-up.
> Both FLIPs scoped batch out, so I'm happy to follow up with a dev@ discussion
> or a FLIP if the community thinks it's needed.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)