weiqingy opened a new pull request, #721:
URL: https://github.com/apache/flink-agents/pull/721
Linked issue: #711
### Purpose of change
PR #657 added short-term memory state TTL but hardcoded the cleanup strategy
to `cleanupFullSnapshot()` in `OperatorStateManager`. This exposes the cleanup
strategy as a public config option
`SHORT_TERM_MEMORY_STATE_TTL_CLEANUP_STRATEGY`, backed by a new public
`ShortTermMemoryTtlCleanupStrategy` enum (same pattern as
`ShortTermMemoryTtlUpdate` / `ShortTermMemoryTtlVisibility`), so users can pick
the strategy that fits their deployment — notably `ROCKSDB_COMPACTION_FILTER`
for long-running RocksDB pipelines on incremental checkpoints. Default
`FULL_SNAPSHOT` preserves prior behavior exactly.
FULL_SNAPSHOT -> cleanupFullSnapshot()
INCREMENTAL -> cleanupIncrementally(5, false)
ROCKSDB_COMPACTION_FILTER -> cleanupInRocksdbCompactFilter(1000L)
LAZY -> disableCleanupInBackground()
The incremental (`5, false`) and RocksDB (`1000L`) parameters reproduce
Flink's internal defaults (which are package-private and not referenceable).
`ROCKSDB_COMPACTION_FILTER` is only effective with the RocksDB state backend;
`LAZY` disables background cleanup so expired entries are reclaimed on read
access only.
### Tests
- Java unit test on the strategy translator, asserting each enum value
produces the correct `StateTtlConfig` cleanup strategy via
`getCleanupStrategies()`.
- Existing `ShortTermMemoryTTLIntegrationTest` still passes (the
`FULL_SNAPSHOT` default is unchanged).
- Python enum round-trip test, plus the cross-language compatibility-test
assertions (key/type/default parity with the Java option).
### API
Yes. Adds one public config option
`AgentExecutionOptions.SHORT_TERM_MEMORY_STATE_TTL_CLEANUP_STRATEGY` and the
public enum `ShortTermMemoryTtlCleanupStrategy` (Java + Python). No Flink types
are exposed in the public surface.
### Documentation
- [ ] `doc-needed`
- [x] `doc-not-needed`
- [ ] `doc-included`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
