[
https://issues.apache.org/jira/browse/SAMZA-1929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei Song updated SAMZA-1929:
----------------------------
Description:
When TTL is set on RocksDB, the default usage of log compacted topic for
changelog isn't appropriate. Ideally, it should match the behavior with a TTL.
So we would adjust changelog topic setting, when
1) TTL is set for RocksDB and
2) Changelog is enabled
3) User has not set them
And we would
1) Disable log compaction on Kafka changelog topic
2) Set topic TTL to be slightly larger than RocksDB TTL
User should be able to override this behavior through
TableDescriptor.withConfig(), e.g.
was:
The changelog Kafka topic level configuration may be useful for adjustment,
when TTL is set on a RocksDB store or when enabled for in-memory DB. User
should be able to set Kafka topic level properties as follows
TableDescriptor desc = new RocksDbTableDescriptor(...)
.withChangelogKafkaTopicLevelProperty("cleanup.policy", "delete")
.withChangelogKafkaTopicLevelProperty("retention.ms", "3600");
> Automatically apply changelog topic level settings, when RocksDB TTL is set
> ---------------------------------------------------------------------------
>
> Key: SAMZA-1929
> URL: https://issues.apache.org/jira/browse/SAMZA-1929
> Project: Samza
> Issue Type: New Feature
> Components: kv-store
> Reporter: Wei Song
> Assignee: Wei Song
> Priority: Major
>
> When TTL is set on RocksDB, the default usage of log compacted topic for
> changelog isn't appropriate. Ideally, it should match the behavior with a TTL.
> So we would adjust changelog topic setting, when
> 1) TTL is set for RocksDB and
> 2) Changelog is enabled
> 3) User has not set them
> And we would
> 1) Disable log compaction on Kafka changelog topic
> 2) Set topic TTL to be slightly larger than RocksDB TTL
> User should be able to override this behavior through
> TableDescriptor.withConfig(), e.g.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
