Alieh Saeedi created KAFKA-21505:
------------------------------------
Summary: Headers-aware store adapters don't propagate
readOnly(IsolationLevel), so READ_COMMITTED reads see uncommitted writes
Key: KAFKA-21505
URL: https://issues.apache.org/jira/browse/KAFKA-21505
Project: Kafka
Issue Type: Bug
Components: streams
Reporter: Alieh Saeedi
When a headers-aware store is built on top of a store in the old (non-headers)
format, the builder inserts a format-converting adapter between the outer
layers and the inner store:
- PlainToHeadersStoreAdapter, TimestampedToHeadersStoreAdapter
(TimestampedKeyValueStoreBuilderWithHeaders)
- PlainToHeadersWindowStoreAdapter, TimestampedToHeadersWindowStoreAdapter
(TimestampedWindowStoreWithHeadersBuilder)
- SessionToHeadersStoreAdapter (SessionStoreWithHeadersBuilder)
None of these adapters override readOnly(IsolationLevel), so they inherit the
interface default, which returns `this`.
With enable.transactional.statestores=true, the outer layers (metered,
change-logging, caching) pass readOnly(level) down to their inner store, and
the RocksDB stores (RocksDBStore, RocksDBWindowStore, RocksDBSessionStore)
override it so that READ_COMMITTED skips the transaction buffer. The chain
stops at the adapter: it returns itself, and reads keep going through the inner
store's transaction buffer. A READ_COMMITTED read therefore sees writes that
have not been committed yet.
Cause: the adapters (KAFKA-20134, KAFKA-20158) were added before
readOnly(IsolationLevel) was introduced for these store types (KAFKA-20496,
KAFKA-20768), and they were not updated when it was.
Affected only when all of these hold:
- enable.transactional.statestores=true
- a headers-aware store is backed by a store in the old format (adapter path)
- the store is read with READ_COMMITTED
Proposed fix: override readOnly(IsolationLevel) in each adapter to call
inner.readOnly(level) and wrap the result in the same format conversion the
adapter applies to normal reads. Then remove readOnly from SPECIAL in
TimestampedToHeadersWindowStoreAdapterCompletenessTest and
SessionToHeadersStoreAdapterCompletenessTest (added in KAFKA-20328,
https://github.com/apache/kafka/pull/22992), and add a test that a
READ_COMMITTED view through each adapter does not see staged writes.
Related: the older timestamped adapters
(KeyValueToTimestampedKeyValueByteStoreAdapter,
WindowToTimestampedWindowByteStoreAdapter) do not override readOnly either and
may have the same gap.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)