[
https://issues.apache.org/jira/browse/FLINK-23426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17383910#comment-17383910
]
Timo Walther commented on FLINK-23426:
--------------------------------------
Hi [~zoucao], we are in the process of supporting updates on all operators.
Grouping windows support them now. Over windows are still on the todo list I
think? We can discuss such feature in a separate issue. The core question here
is whether it should be possible to use stream operators also in batch mode.
Because as far as I know, we currently don't support Windowing table-valued
functions, Temporal Joins or MATCH_RECOGNIZE in batch.
> Support changelog processing in batch mode
> ------------------------------------------
>
> Key: FLINK-23426
> URL: https://issues.apache.org/jira/browse/FLINK-23426
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / API
> Reporter: Timo Walther
> Priority: Major
>
> The DataStream API can execute arbitrary DataStream programs when running in
> batch mode. However, this is not the case for the Table API batch mode. E.g.
> a source with non-insert only changes is not supported and updates/deletes
> cannot be emitted.
> In theory, we could make this work by running the "stream mode" of the
> planner (CDC transformations) on top of the "batch mode" of DataStream API
> (specialized state backend, sorted inputs). It is up for discussion if and
> how we expose such functionality.
> If we don't allow enabling incremental updates, we can also add a special
> batch operator that materializes the incoming changes for a batch pipeline.
> However, it would require "complete" CDC logs (i.e. no missing UPDATE_AFTER).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
