vcrfxia opened a new pull request, #13243: URL: https://github.com/apache/kafka/pull/13243
(This PR is stacked on https://github.com/apache/kafka/pull/13189. Only the last commit needs to be reviewed separately.) The RocksDB-based implementation for versioned key-value stores introduced in https://github.com/apache/kafka/pull/13188 has a well-defined "history retention" parameter which specifies how far back in time (relative to the current observed stream time) reads may take place, but there is no well-defined equivalent (aka "grace period") for how far back in time writes will be accepted. Instead, there is an implicit grace period whereby the store accepts all writes which affect valid reads. This doesn't quite work, though, because it requires infinite tombstone retention when the latest value for a particular key is a tombstone -- if the latest value for a key is a very old tombstone, we can’t expire it because if there’s an even older non-null put to store later, then without the tombstone we’ll accept this write as the latest value for the key, even though it isn't. In light of this, this PR changes the versioned store semantics to define an explicit "grace period" property. ([KIP-889](https://cwiki.apache.org/confluence/display/KAFKA/KIP-889%3A+Versioned+State+Stores) has been updated accordingly.) For now, grace period will always be equal to the history retention, though in the future we can introduce a new KIP to expose options to configure grace period separately. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
