[
https://issues.apache.org/jira/browse/FLINK-24666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weijie Guo updated FLINK-24666:
-------------------------------
Affects Version/s: 2.1.0
(was: 1.14.0)
(was: 1.13.3)
> Add job level "table.exec.state-stale.error-handling" option and apply to
> related stateful stream operators
> -----------------------------------------------------------------------------------------------------------
>
> Key: FLINK-24666
> URL: https://issues.apache.org/jira/browse/FLINK-24666
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Runtime
> Affects Versions: 2.1.0
> Reporter: lincoln lee
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.0.0
>
>
> In stream processing, records will be deleted when exceed state ttl (if
> configured), and when the corresponding record's update arrives again, the
> operator may not be able to handle it properly, we need a unified error
> handling mechanism to handle this situation,
> instead of each stateful operator currently handling its own.
> e.g., currently, there's a lenient option which was not exposed to users on
> RetractableTopNFunction
> {quote}// flag to skip records with non-exist error instead to fail, true by
> default.
> private final boolean lenient = true
> {quote}
> So there's no chance to raise the exception when the record(s) unexpectedly
> cleared by state ttl. Commonly this happens because a too shorter ttl at Line
> 190 or inconstancy between the two internal state(dataState and treeMap) at
> other place.
> {quote}List<RowData> inputs = dataState.get(key);
> if (inputs == null)
> Unknown macro: \{ // Skip the data if it's state is cleared because of state
> ttl. if (lenient) Unknown macro}
> else
> Unknown macro: \{ throw new RuntimeException(STATE_CLEARED_WARN_MSG); }
> }
> {quote}
> We'd better to expose it to users (default value can be true to keep
> consistent with previous version) and should be unified to cover all the
> stateful stream operators than specific to RetractableTopNFunction.
> The new option 'table.exec.state-stale.error-handling' has three values:
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
