[
https://issues.apache.org/jira/browse/FLINK-33183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844094#comment-17844094
]
lincoln lee commented on FLINK-33183:
-------------------------------------
[~martijnvisser] Thanks for the reminder and sorry for my late response! I've
thought about this issue many times, but I still think determine a column's
determinism by whether a column is virtual or not is too subtle, also I don't
want to introduce a new option(which maybe difficult for users to understand)
to control the logic of determining here. Perhaps what we need more is a way to
deal with the challenges posed by metadata columns, but I don't have a perfect
solution yet.
> Enable metadata columns in NduAnalyzer with retract if non-virtual
> ------------------------------------------------------------------
>
> Key: FLINK-33183
> URL: https://issues.apache.org/jira/browse/FLINK-33183
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / Planner
> Reporter: Timo Walther
> Priority: Major
>
> Currently, the NduAnalyzer is very strict about metadata columns in updating
> sources. Compared to append and upsert sources (see also FLINK-33182),
> retract sources are tricky. And the analyzer is actually correct.
> However, for retract sources we should expose more functionality to the user
> and add a warning to the documentation that retract mode could potentially
> cause NDU problems if not enough attention is paid. We should only throw an
> error on virtual metadata columns. Persisted metadata columns can be
> considered “safe“. When a metadata column is persisted, we can assume that an
> upstream Flink job fills its content thus likely also fills its correct
> retraction.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
