Andrey Zagrebin created FLINK-12005:
---------------------------------------
Summary: [State TTL] Event time support
Key: FLINK-12005
URL: https://issues.apache.org/jira/browse/FLINK-12005
Project: Flink
Issue Type: New Feature
Reporter: Andrey Zagrebin
Assignee: Yu Li
The event time is opted for in StateTtlConfig by setting
TtlTimeCharacteristic.EventTime.
To enable event time support, the updated watermark needs to be passed to the
state backend, shared with TTL state wrappers and additional cleanup strategies
(snapshot transformers and compaction filter).
h3. Event time provider
Additional implementation of TtlTimeProvider, which holds current watermark,
needs to be passed to the state backend at the moment of its creation in
StreamTaskStateInitializerImpl. There several ways to update watermark in this
implementation of TtlTimeProvider:
* in InternalTimeServiceManager.advanceWatermark explicitly
* InternalTimeServiceManager/InternalTimerServiceImpl could be refactored to
use shared EventTimeService which holds current updatable watermark and wrapped
by TtlTimeProvider
The TTL state wrapping factory should create TTL state wrappers and snapshot
transformers with TtlTimeProvider selected by TtlTimeCharacteristic.
h3. RocksDB TTL compaction filter
The RocksDB TTL compaction filter factory needs to get selected TtlTimeProvider
when it gets configured. There are two ways:
* make it volatile and settable in
RocksDbTtlCompactFiltersManager.TimeProviderWrapper, track it in
RocksDbTtlCompactFiltersManager along with FlinkCompactionFilterFactory to
configure later before configuring FlinkCompactionFilterFactory.
* Move FlinkCompactionFilter.TimeProvider from FlinkCompactionFilterFactory to
ConfigHolder and set selected TtlTimeProvider with the Config.
The second option does not use volatile variable and should be more performant
but needs changing RocksDB java client and either releasing new version
FRocksDB or Flink RocksDB extensions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
