[
https://issues.apache.org/jira/browse/FLINK-11139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16720833#comment-16720833
]
Hequn Cheng commented on FLINK-11139:
-------------------------------------
[~zhaoshijie] Do you mean you have already switched to state ttl on your own
flink version? Could you cherry pick the latest improvement of non-window join
to your own version? The commit is
[link|https://github.com/apache/flink/commit/5716e4d9f64f957faeebd28647ccf3229598f0a4](FLINK-10543).
You have to notice that the state will not be compatible.
The oom problem may not benefit from the delete feature compared to the old
version, because the previous non-window join register timers in a fixed
interval, i.e., only register a new timer when the old timer is fired. However,
the latest improvement should half the number of timers since it uses one
ValueState to control clean up instead of two, while before, left and right
register timers individually. It worth to give it a try. [~zhaoshijie].
Furthermore, you can reduce the value of max retention time, the problem can
also be alleviated(both in old and new flink version).
Even the new improvement can half the number of timers for join. It will still
OOM if the number of keys is big. In the long term, we need to adapt our timer
logic in flink-table to state ttl? What do you think? [~fhueske]
> stream non window join support state ttl
> ----------------------------------------
>
> Key: FLINK-11139
> URL: https://issues.apache.org/jira/browse/FLINK-11139
> Project: Flink
> Issue Type: Improvement
> Components: Table API & SQL
> Affects Versions: 1.7.0
> Reporter: zhaoshijie
> Priority: Major
>
> stream non window join function use timer to delete expired data，it is ok for
> small amount of data or short expiration time，but it will be OOM（too many
> timer）on taskManger when there is a long expiration time and a large
> amount of data。In fact, table module other state function has same problem，I
> would like to contribute to fix it。
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
