[ 
https://issues.apache.org/jira/browse/FLINK-41549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gustavo de Morais resolved FLINK-41549.
---------------------------------------
    Fix Version/s: 2.5.0
       Resolution: Fixed

> FLIP-621: Add built-in TO_APPEND PTF
> ------------------------------------
>
>                 Key: FLINK-41549
>                 URL: https://issues.apache.org/jira/browse/FLINK-41549
>             Project: Flink
>          Issue Type: New Feature
>          Components: Table SQL / API, Table SQL / Planner
>            Reporter: Gustavo de Morais
>            Assignee: Gustavo de Morais
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 2.5.0
>
>
> Many queries produce an updating result (upsert sources, deduplication, 
> aggregations, joins), but the place they write to only accepts appends: an 
> append-only Kafka topic, an append-only sink, or an operator that requires 
> insert-only input. Since {{TO_CHANGELOG}} (FLIP-564) was released, one use 
> case keeps coming up: write inserts and updates to an append-only log and 
> drop deletes. {{TO_CHANGELOG}} can do this, but it requires an {{op_mapping}} 
> that drops deletes plus a projection that removes the op column.
> This FLIP adds a built-in {{TO_APPEND}} process table function that converts 
> an updating table into an append-only table in one call. INSERT and 
> UPDATE_AFTER rows are emitted as INSERT rows, UPDATE_BEFORE and DELETE rows 
> are dropped, and the output keeps the input schema. The function is stateless 
> and has row semantics. With FLINK-41371 and FLINK-41516, an upsert input with 
> an upsert key is consumed as emitted, without a {{ChangelogNormalize}}.
> {code:sql}
> -- Input: upsert table 'customers' with primary key 'id'
> -- +U[id:'c1', city:'Berlin']
> -- +U[id:'c1', city:'Munich']
> -- -D[id:'c1']
> SELECT * FROM TO_APPEND(input => TABLE customers)
> -- Output (append-only):
> -- +I[id:'c1', city:'Berlin']
> -- +I[id:'c1', city:'Munich']
> {code}
> The Table API exposes the same function as {{Table.toAppend()}}.
> FLIP: 
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-621%3A+Add+built-in+TO_APPEND+PTF
> Discussion thread: 
> https://lists.apache.org/thread/g5gytjmlfhglokl7zpcfqd6xh70snjb8
> Vote thread: https://lists.apache.org/thread/5lof8ms2s2ycn090ozjzvz1jqfcs7z7m



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to