GitHub user PragmaTwice edited a comment on the discussion: TimeSeries Proposal
Some comments: - In Kvrocks there are several existing column families. And the most relevant CFs here are `metadata` (metadata CF) and `default` (subkey CF), where all keys inside should follow a special key format (user keys and sub keys, per https://kvrocks.apache.org/community/data-structure-on-rocksdb#user-key-encoding). I think it is clear that "Datameta" in the proposal is stored in `metadata` CF, but for other parts we can specify clearly whether it is stored in subkey CF or a new timeseries CF. - There may be some pros and cons for introducing a new CF (e.g. we face some [compatibility issue of introducing one](https://github.com/apache/kvrocks/discussions/2851)). If we introduce a CF for timeseries, we should consider: - How to avoid any key conflicts? e.g. in the proposal, there may be some conflicts since we didn't introduce some enum values as prefixes in the key format (for example, a key in "Reverse Index" format may be same as another key in "Label Index"). Also, we cannot iterate over a special type of keys if we don't distingush them by prefixing keys with some enum values. (Maybe you can find some idea in the encoding of search data: https://kvrocks.apache.org/community/kvrocks-search-index-encoding.) - How to do garbage-collection? We have key expiration and also timeseries-specific "retention", and we need to modify compaction filter (https://github.com/apache/kvrocks/blob/unstable/src/storage/compact_filter.cc) and maybe also compaction checker (https://github.com/apache/kvrocks/blob/unstable/src/storage/compaction_checker.cc) to figure out (1): which key is expired if a compaction is triggered, and (2): when we need to trigger a compaction (maybe there's timeseries specific logic). - Also I found some reference to codebase of redis timeseries. We can of course read its code, but we cannot use them directly (i.e. copy ANY code from its codebase) due to ASF license requirements (it's AGPLv3 licensed). GitHub link: https://github.com/apache/kvrocks/discussions/3044#discussioncomment-13659303 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
