[
https://issues.apache.org/jira/browse/KAFKA-20413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alieh Saeedi updated KAFKA-20413:
---------------------------------
Description:
PR [#21969 |https://github.com/apache/kafka/pull/21950]added header propagation
to serialize/deserialize calls in InMemoryTimeOrderedKeyValueChangeBuffer and
RocksDBTimeOrderedKeyValueBuffer, but did not implement header-aware store
variants (similar to how TimestampedKeyValueStore vs
TimestampedKeyValueStoreWithHeaders exist).
Per PR comments: The team initially thought time-ordered buffers could be
skipped because they're only used for stream-table join with versioned tables
(scheduled for 4.4), but forgot that InMemoryTimeOrderedKeyValueChangeBuffer is
used by suppress().
Missing Components
1. Suppress buffer - No header-aware store variant
- InMemoryTimeOrderedKeyValueChangeBuffer needs headers-aware store
infrastructure
- Currently only passes headers to serialize/deserialize (PR #21969) but
lacks proper store variant
- Used by: suppress() operator
was:
PR [#21969 |https://github.com/apache/kafka/pull/21950]added header propagation
to serialize/deserialize calls in InMemoryTimeOrderedKeyValueChangeBuffer and
RocksDBTimeOrderedKeyValueBuffer, but did not implement header-aware store
variants (similar to how TimestampedKeyValueStore vs
TimestampedKeyValueStoreWithHeaders exist).
Per PR comments: The team initially thought time-ordered buffers could be
skipped because they're only used for stream-table join with versioned tables
(scheduled for 4.4), but forgot that InMemoryTimeOrderedKeyValueChangeBuffer is
used by suppress().
Missing Components
1. Suppress buffer - No header-aware store variant
- InMemoryTimeOrderedKeyValueChangeBuffer needs headers-aware store
infrastructure
- Currently only passes headers to serialize/deserialize (PR #21969) but
lacks proper store variant
- Used by: suppress() operator
2. ListValueStore - No header support at all
- ListValueStore.java: Passes null/no headers to LIST_SERDE
- ChangeLoggingListValueBytesStore.java: Uses new RecordHeaders() instead of
actual headers
- No headers-aware builder variant
- Used by: stream-stream left/outer joins
> Missing header-aware store variants for suppress()
> ---------------------------------------------------
>
> Key: KAFKA-20413
> URL: https://issues.apache.org/jira/browse/KAFKA-20413
> Project: Kafka
> Issue Type: Sub-task
> Components: streams
> Reporter: Alieh Saeedi
> Assignee: Alieh Saeedi
> Priority: Minor
>
> PR [#21969 |https://github.com/apache/kafka/pull/21950]added header
> propagation to serialize/deserialize calls in
> InMemoryTimeOrderedKeyValueChangeBuffer and RocksDBTimeOrderedKeyValueBuffer,
> but did not implement header-aware store variants (similar to how
> TimestampedKeyValueStore vs TimestampedKeyValueStoreWithHeaders exist).
>
>
>
> Per PR comments: The team initially thought time-ordered buffers could be
> skipped because they're only used for stream-table join with versioned tables
> (scheduled for 4.4), but forgot that InMemoryTimeOrderedKeyValueChangeBuffer
> is used by suppress().
>
>
>
> Missing Components
> 1. Suppress buffer - No header-aware store variant
>
>
>
> - InMemoryTimeOrderedKeyValueChangeBuffer needs headers-aware store
> infrastructure
>
> - Currently only passes headers to serialize/deserialize (PR #21969) but
> lacks proper store variant
> - Used by: suppress() operator
>
>
>
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
