[
https://issues.apache.org/jira/browse/FLINK-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17031663#comment-17031663
]
Andrey Zagrebin edited comment on FLINK-15938 at 2/6/20 3:42 PM:
-----------------------------------------------------------------
[~libenchao]
For 1.9:
You are right that, by default, the expired state cleanup will happen only
when the state is read explicitly by the user code.
We were conservative here to avoid potential problems with the new background
cleanup feature.
You can activate the background cleanup in 1.9:
{code:java}
return StateTtlConfig
...
.cleanupInBackground()
.build();{code}
Then, you do not have to read the expired state explicitly, it should be
cleaned in background. Read also docs further about the background cleanup.
For 1.10, we decided to activate the background cleanup by default but there
was no plan to do it for 1.9. Is it what you wanted to resolve with this issue?
was (Author: azagrebin):
[~libenchao]
For 1.9:
You are right that, by default, the expired state cleanup will happen only
when the state is read explicitly by the user code.
We were conservative here to avoid potential problems with the new background
cleanup feature.
You can activate the background cleanup in 1.9:
{code:java}
return StateTtlConfig
...
.cleanupInBackground()
.build();{code}
Then, you do not have to read the expired state explicitly, it should be
cleaned in background. Read also docs further about the background cleanup.
For 1.10, we decided to activate the background cleanup by default but there
was no plan to do it for 1.9. Is it what you wanted to resolve with this issue?
If so, I suggest to close this issue.
> Idle state not cleaned in StreamingJoinOperator and
> StreamingSemiAntiJoinOperator
> ---------------------------------------------------------------------------------
>
> Key: FLINK-15938
> URL: https://issues.apache.org/jira/browse/FLINK-15938
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
> Affects Versions: 1.9.2
> Reporter: Benchao Li
> Priority: Critical
>
> {code:java}
> return StateTtlConfig
> .newBuilder(Time.milliseconds(retentionTime))
> .setUpdateType(StateTtlConfig.UpdateType.OnCreateAndWrite)
>
> .setStateVisibility(StateTtlConfig.StateVisibility.ReturnExpiredIfNotCleanedUp)
> .build();
> {code}
> StateTtl is constructed in above code for `StreamingJoinOperator` and
> `StreamingSemiAntiJoinOperator`.
> However, as stated in
> [https://ci.apache.org/projects/flink/flink-docs-master/dev/stream/state/state.html#cleanup-of-expired-state]
> , the state will be cleaned only when it's read which means the state will
> not be cleaned enless we read it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
