[
https://issues.apache.org/jira/browse/FLINK-41549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18148599#comment-18148599
]
Gustavo de Morais commented on FLINK-41549:
-------------------------------------------
Fixed in apache/flink:master 28d3164e03033e23aa1fb73479b4d1897af56314
> FLIP-621: Add built-in TO_APPEND PTF
> ------------------------------------
>
> Key: FLINK-41549
> URL: https://issues.apache.org/jira/browse/FLINK-41549
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / API, Table SQL / Planner
> Reporter: Gustavo de Morais
> Assignee: Gustavo de Morais
> Priority: Major
> Labels: pull-request-available
>
> Many queries produce an updating result (upsert sources, deduplication,
> aggregations, joins), but the place they write to only accepts appends: an
> append-only Kafka topic, an append-only sink, or an operator that requires
> insert-only input. Since {{TO_CHANGELOG}} (FLIP-564) was released, one use
> case keeps coming up: write inserts and updates to an append-only log and
> drop deletes. {{TO_CHANGELOG}} can do this, but it requires an {{op_mapping}}
> that drops deletes plus a projection that removes the op column.
> This FLIP adds a built-in {{TO_APPEND}} process table function that converts
> an updating table into an append-only table in one call. INSERT and
> UPDATE_AFTER rows are emitted as INSERT rows, UPDATE_BEFORE and DELETE rows
> are dropped, and the output keeps the input schema. The function is stateless
> and has row semantics. With FLINK-41371 and FLINK-41516, an upsert input with
> an upsert key is consumed as emitted, without a {{ChangelogNormalize}}.
> {code:sql}
> -- Input: upsert table 'customers' with primary key 'id'
> -- +U[id:'c1', city:'Berlin']
> -- +U[id:'c1', city:'Munich']
> -- -D[id:'c1']
> SELECT * FROM TO_APPEND(input => TABLE customers)
> -- Output (append-only):
> -- +I[id:'c1', city:'Berlin']
> -- +I[id:'c1', city:'Munich']
> {code}
> The Table API exposes the same function as {{Table.toAppend()}}.
> FLIP:
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-621%3A+Add+built-in+TO_APPEND+PTF
> Discussion thread:
> https://lists.apache.org/thread/g5gytjmlfhglokl7zpcfqd6xh70snjb8
> Vote thread: https://lists.apache.org/thread/5lof8ms2s2ycn090ozjzvz1jqfcs7z7m
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)