[
https://issues.apache.org/jira/browse/FLINK-9938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565233#comment-16565233
]
ASF GitHub Bot commented on FLINK-9938:
---------------------------------------
StefanRRichter commented on a change in pull request #6460: [FLINK-9938] Clean
up full snapshot from expired state with TTL
URL: https://github.com/apache/flink/pull/6460#discussion_r206838305
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/state/heap/HeapKeyedStateBackend.java
##########
@@ -276,16 +277,19 @@ private boolean hasRegisteredState() {
}
@Override
+ @Nonnull
public <N, SV, S extends State, IS extends S> IS createInternalState(
- TypeSerializer<N> namespaceSerializer,
- StateDescriptor<S, SV> stateDesc) throws Exception {
+ @Nonnull TypeSerializer<N> namespaceSerializer,
+ @Nonnull StateDescriptor<S, SV> stateDesc,
+ @Nonnull StateSnapshotFilter<SV> snapshotFilter) throws
Exception {
StateFactory stateFactory =
STATE_FACTORIES.get(stateDesc.getClass());
if (stateFactory == null) {
String message = String.format("State %s is not
supported by %s",
stateDesc.getClass(), this.getClass());
throw new FlinkRuntimeException(message);
}
StateTable<K, N, SV> stateTable =
tryRegisterStateTable(namespaceSerializer, stateDesc);
+ stateTable.setSnapshotFilter(snapshotFilter);
Review comment:
I wonder if this should even hang around freely in the table. It feels like
it `RegisteredKeyValueStateBackendMetaInfo` might be the closest place to store
those filter because it has anything "meta" about a key-value state. It exists
also in heap and rocks and should be available in all places that could ask for
a filter.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> State TTL cleanup during full state scan upon checkpointing
> -----------------------------------------------------------
>
> Key: FLINK-9938
> URL: https://issues.apache.org/jira/browse/FLINK-9938
> Project: Flink
> Issue Type: Improvement
> Components: State Backends, Checkpointing
> Affects Versions: 1.6.0
> Reporter: Andrey Zagrebin
> Assignee: Andrey Zagrebin
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.6.0
>
>
> We can try to piggyback full state scan during certain checkpoint processes
> in backends, check TTL expiration for every entry and evict expired to speed
> up cleanup.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
