vcrfxia commented on code in PR #13444:
URL: https://github.com/apache/kafka/pull/13444#discussion_r1154765044
##########
docs/streams/developer-guide/processor-api.html:
##########
@@ -396,12 +400,68 @@ <h2>
</ul>
</p>
</div>
+ <div class="section" id="versioned-state-stores">
+ <span
id="streams-developer-guide-state-store-versioned"></span><h3><a
class="toc-backref" href="#id12">Versioned Key-Value State Stores</a><a
class="headerlink" href="#versioned-state-stores" title="Permalink to this
headline"></a></h3>
+ <p>Versioned key-value state stores are available since Kafka
Streams 3.5.
+ Rather than storing a single record version (value and
timestamp) per key,
+ versioned state stores may store multiple record versions
per key. This
+ allows versioned state stores to support timestamped
retrieval operations
+ to return the latest record (per key) as of a specified
timestamp, which
+ enables use cases such as temporal stream-table joins.</p>
+ <p>You can create a persistent, versioned state store by
passing a
+ <a class="reference external"
href="/{{version}}/javadoc/org/apache/kafka/streams/state/Stores.html#persistentVersionedKeyValueStore(java.lang.String,java.time.Duration)">VersionedBytesStoreSupplier</a>
+ to the
+ <a class="reference external"
href="/{{version}}/javadoc/org/apache/kafka/streams/state/Stores.html#versionedKeyValueStoreBuilder(java.lang.String,java.time.Duration)">versionedKeyValueStoreBuilder</a>,
+ or by implementing your own
+ <a class="reference external"
href="/{{version}}/javadoc/org/apache/kafka/streams/state/VersionedKeyValueStore.html">VersionedKeyValueStore</a>.</p>
+ <p>Each versioned store has an associated, fixed-duration
<em>history retention</em>
+ parameter which specifies long old record versions should
be kept for.
+ In particular, a versioned store guarantees to return
accurate results for
+ timestamped retrieval operations where the timestamp being
queried is within
+ history retention of the current observed stream time.</p>
+ <p>History retention also doubles as its <em>grace
period</em>, which determines
+ how far back in time out-of-order writes to the store will
be accepted. A
+ versioned store will not accept writes (inserts, updates,
or deletions) if
+ the timestamp associated with the write is older than the
current observed
+ stream time by more than the grace period. Stream time in
this context is
+ tracked per-partition, rather than per-key, which means
it's important
+ that grace period (i.e., history retention) be set high
enough to
+ accommodate a record with one key arriving out-of-order
relative to a
+ record for another key.</p>
+ <p>Because the memory footprint of versioned key-value stores
is higher than
+ that of non-versioned key-value stores, you may want to
adjust your
+ <a class="reference internal"
href="memory-mgmt.html#streams-developer-guide-memory-management-rocksdb"><span
class="std std-ref">RocksDB memory settings</span></a>
+ accordingly. Benchmarking your application with versioned
stores is also
+ advised as performance is expected to be worse than when
using non-versioned
+ stores.</p>
+ <p>Versioned stores do not support caching or interactive
queries at this time.
+ Also, window stores may not be versioned.</p>
+ <b>Upgrade note:</b> Versioned state stores are opt-in only;
no automatic
Review Comment:
I've added a section into the upgrade guide which links to the developer
guide section. I think this is a good compromise in order to keep the upgrade
guide smaller, and to be consistent with the fact that the detailed upgrade
note for timestamped key-value stores is also in the developer guide (rather
than the upgrade guide).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
