Thanks for the feedback! Following the discussion about using Record as the result type, we're refining the result model. Instead of returning the PAPI Record, IQ header queries return a new read-only interface ReadOnlyRecord (key/value/timestamp/headers), which Record implements.
Reason: returning Record directly exposes its withKey/withValue/… builders, which are meant for transform-and-forward in processing and do nothing on a read-only query result (a false affordance). Conceptually, an IQ result and a Record are the same read concept seen from two layers: ReadOnlyRecord is the shared read view, and Record extends it with transform-and-forward. The change to PAPI is additive only. Record already has those accessors, so it just declares implements ReadOnlyRecord (source/binary compatible, no behavior change). Thanks - Jess On Wed, Jun 17, 2026 at 5:59 PM Alieh Saeedi via dev <[email protected]> wrote: > Hey Matthias > > We store data in SSs in key value form. So the queries return value V which > is ValueTimestampHeaders in headers-aware SSs case (and ValueAndTimestamp > in timestamped SSs). Does returning Record mean that we consider V as an > intermittent result and then we convert it to Record for the user? > > Thanks > > -Alieh > > On Tue, Jun 16, 2026, 8:28 PM Matthias J. Sax <[email protected]> wrote: > > > Thanks. > > > > I just had a high level thought. Not sure if this idea is good or bad, > > so I just wanted to put it out, to see what people think. > > > > With the "new" PAPI, we move from KV to `Record` model. For IQ, we > > historically also use a KV-based design, and extended it incrementally > > via `ValueAndTimestamp` and most recently via `ValueTimestampHeaders`. > > > > While we need `ValueTimestampHeaders` internally (it's the par that goes > > into the RocksDB value), I am wondering if it could make sense for IQv2 > > to also use `Record`? > > > > If we use `Record`, we would avoid the `ValueTimestampHeaders` vs > > `AggregationsWithHeaders` (which also have a ts, but just stored in the > > RocksDB `SessionKey` object) question, and long term, we might also move > > to a `Record` model in the DSL. So we could align and standardize on > > `Record` throughout the whole API surface area in the long term? > > > > > > Thoughts? > > > > > > -Matthias > > > > On 6/16/26 7:19 AM, Jess Jin via dev wrote: > > > Thanks Alieh! Updated in the wiki. > > > > > > - Jess > > > > > > On Tue, Jun 16, 2026 at 7:49 AM Alieh Saeedi via dev < > > [email protected]> > > > wrote: > > > > > >> Hey Jess, > > >> I see that you agreed with some of the suggestions and design > > alternatives, > > >> including the naming. Could you please: > > >> > > >> - Update the wiki to the committed design? > > >> > > >> > > >> - Fix the motivation wording: existing query types don't "drop" > > headers... > > >> > > >> Thanks > > >> - Alieh > > >> > > >> On Mon, Jun 15, 2026 at 10:02 PM Jess Jin via dev < > [email protected] > > > > > >> wrote: > > >> > > >>> Hi Lucas, > > >>> > > >>> Thanks! I agree. The new type only offers withKey(...) (single key → > > all > > >>> sessions for that key), so "Range" is misleading. I'll rename it to > > >>> SessionKeyWithHeadersQuery. That also keeps the "Range" name free > for a > > >>> future genuine key-range session query. > > >>> > > >>> - Jess > > >>> > > >>> On Mon, Jun 15, 2026 at 11:03 AM Lucas Brutschy < > > [email protected]> > > >>> wrote: > > >>> > > >>>> Hi Jess, > > >>>> > > >>>> Thanks for the KIP, and for working through the latest round with > > >>>> Matthias and Bill. > > >>>> > > >>>> LB1: For the session query, the existing IQv2 entry point is > > >>>> WindowRangeQuery created via withKey(), which returns all sessions > for > > >>>> a single key. Is "Range" the right word when only withKey() is > > >>>> offered? Seems "Range" previously referred to a range of keys, but > > >>>> that range is not part of the new type anymore. > > >>>> > > >>>> Thanks, > > >>>> Lucas > > >>>> > > >>>> On Mon, Jun 15, 2026 at 4:37 PM Jess Jin via dev < > > [email protected] > > >>> > > >>>> wrote: > > >>>>> > > >>>>> Hi Bill, > > >>>>> > > >>>>> Thanks for the review. I agree this needed to be nailed down. > > >>>>> > > >>>>> - Jess > > >>>>> > > >>>>> On Fri, Jun 12, 2026 at 2:41 PM Bill Bejeck <[email protected]> > > >> wrote: > > >>>>> > > >>>>>> Hi Jess, > > >>>>>> > > >>>>>> Thanks for the KIP. > > >>>>>> > > >>>>>> I'd second the importance of getting the details flushed out in > > >>>> Matthias's > > >>>>>> 4th comment with regards to session stores > > >>>>>> and how we'll support `ValueTimestampeHeader` and > > >>>> `AggregationWithHeaders` > > >>>>>> and if session-header-stores are supported in this KIP. > > >>>>>> > > >>>>>> -Bill > > >>>>>> > > >>>>>> On Thu, Jun 11, 2026 at 8:59 PM Matthias J. Sax <[email protected] > > > > >>>> wrote: > > >>>>>> > > >>>>>>> Thanks for the KIP. Very nice to see that IQv2 gets more love. > > >>>>>>> > > >>>>>>> > > >>>>>>> Couple of comments/questions. > > >>>>>>> > > >>>>>>> > > >>>>>>> MJS1: The KIP says. > > >>>>>>> > > >>>>>>>> surfaced through the public wrapper type > > >> ValueTimestampHeaders<V> > > >>>>>>> > > >>>>>>> Yes, but session-header-store actually uses > > >>> `AggregationWithHeaders`. > > >>>>>>> Should we mention both cases? Or do we not include > > >> session-headers > > >>>>>>> stores? Later, the KIP only mentions > > >>>>>>> `MeteredTimestampedKeyValueStoreWithHeaders` and > > >>>>>>> `MeteredTimestampedWindowStoreWithHeaders`, so maybe that's > > >> indeed > > >>>> what > > >>>>>>> is proposed? If yes, why do exclude session-header-store? Also > > >> if, > > >>>> yes, > > >>>>>>> would be good to call this out explicitly. > > >>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> MJS2: The KIP says. > > >>>>>>> > > >>>>>>>> The existing query types — TimestampedKeyQuery, > > >>>> TimestampedRangeQuery, > > >>>>>>> WindowKeyQuery, and WindowRangeQuery — all return values without > > >>>> headers, > > >>>>>>> even when run against a header-aware store. > > >>>>>>> > > >>>>>>> This sound as if these query types are already supported, but I > > >>>> believe > > >>>>>>> they are not. Same for their timestamp-less equivalents like > > >>>> `KeyQuery`. > > >>>>>>> > > >>>>>>> I think it would be a good extension to the KIP, to also add > > >>> support > > >>>> for > > >>>>>>> these existing query types on the newly added header-stores. We > > >>> don't > > >>>>>>> need to define any new classed. The KIP would just say, that we > > >>>> extend > > >>>>>>> the kv-ts-header store to also support existing KeyQuery and > > >>>>>>> TimestampedKeyQuery (and similar for window-header-store; and > > >> maybe > > >>>> also > > >>>>>>> session-header-store, in case the KIP does include > > >>>> session-header-stores) > > >>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> MJS3: Question about naming. You propose to add: > > >>>>>>> > > >>>>>>> - TimestampedKeyWithHeadersQuery > > >>>>>>> - TimestampedRangeWithHeadersQuery > > >>>>>>> - WindowKeyWithHeadersQuery > > >>>>>>> - WindowRangeWithHeadersQuery > > >>>>>>> > > >>>>>>> But all four query types return some form of > > >>> `ValueTimestampHeaders` > > >>>>>>> results. So it seems we should align the names, and maybe call > > >> the > > >>>> later > > >>>>>>> two > > >>>>>>> > > >>>>>>> - TimestampedWindowKeyWithHeadersQuery > > >>>>>>> - TimestampedWindowRangeWithHeadersQuery > > >>>>>>> > > >>>>>>> Or, to avoid very long names, shorten the first two to > > >>>>>>> > > >>>>>>> - KeyWithHeadersQuery > > >>>>>>> - RangeWithHeadersQuery > > >>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> MJS4: About `WindowRangeWithHeadersQuery`. > > >>>>>>> > > >>>>>>> First some background: The existing `WindowRangeQuery` has two > > >>>> methods > > >>>>>>> how to setup the query, and depending which one is used, the > > >> query > > >>>> can > > >>>>>>> be used against either a windowed-store or a session-store, ie, > > >>> while > > >>>>>>> both are a `WindowRangeQuery`, it's technically two different > > >>> queries > > >>>>>>> for two different stores. > > >>>>>>> > > >>>>>>> - `withKey(...)` -> only supported by session stores > > >>>>>>> - `withWindowStartRange(...)` -> only supported by window > store > > >>>>>>> > > >>>>>>> The KIP proposed to add both methods, too, but at the same time > > >>> says, > > >>>>>>> that window-store won't accept a query created via `withKey` > > >> (what > > >>> in > > >>>>>>> general aligns to what we have). But the KIP seems to exclude > > >>>>>>> session-header-store support. So why would we add `withKey` at > > >> all? > > >>>>>>> Would it not be simpler to only add the supported > > >>>> `withWindowStartRange` > > >>>>>>> method? > > >>>>>>> > > >>>>>>> Of course, if we want to mimic the existing query type, the > > >>>>>>> WindowRangeWithHeadersQuery seems to face the challenge that > > >>>>>>> window-header-store uses `ValueTimestampeHeader` while > > >>>>>>> session-header-stores uses `AggregationWithHeaders` type. I would > > >>>> like > > >>>>>>> to understand better how this should all fit together. Also for > > >> the > > >>>>>>> case, that we add session-header-store support later (ie, we > > >> should > > >>>>>>> avoid to corner ourselves and ensure what we define now, is > > >>>> extensible > > >>>>>>> in the future). > > >>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> -Matthias > > >>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> On 6/11/26 6:23 AM, Jess Jin via dev wrote: > > >>>>>>>> Hey Alieh, > > >>>>>>>> > > >>>>>>>> Thanks for the feedback! Updated in the wiki page. > > >>>>>>>> > > >>>>>>>> - Jess > > >>>>>>>> > > >>>>>>>> On Wed, Jun 10, 2026 at 3:26 PM Alieh Saeedi < > > >>> [email protected] > > >>>>> > > >>>>>>> wrote: > > >>>>>>>> > > >>>>>>>>> Hey Jess, > > >>>>>>>>> > > >>>>>>>>> Thanks for putting together the KIP. > > >>>>>>>>> > > >>>>>>>>> A couple of small nits: > > >>>>>>>>> > > >>>>>>>>> AS1: Please replace the `getX()` methods with `x()`. For > > >>> example, > > >>>>>>>>> `getKey()` should become `key()`. As far as I know, that’s the > > >>>> usual > > >>>>>>>>> convention. > > >>>>>>>>> AS2: In the `Usage Example` section, please replace `long` > > >> with > > >>>>>> `Long`, > > >>>>>>>>> since `ValueTimestampHeaders.value` is not a primitive. > > >>>>>>>>> > > >>>>>>>>> - Alieh > > >>>>>>>>> > > >>>>>>>>> On Wed, Jun 10, 2026 at 7:02 PM Jess Jin via dev < > > >>>>>> [email protected]> > > >>>>>>>>> wrote: > > >>>>>>>>> > > >>>>>>>>>> Hi all, > > >>>>>>>>>> > > >>>>>>>>>> I'd like to start a discussion on KIP-1356: Introduce IQv2 > > >> for > > >>>>>>>>>> headers-aware state stores. Link > > >>>>>>>>>> < > > >>>>>>>>>> > > >>>>>>> > > >>>>>> > > >>>> > > >>> > > >> > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/KAFKA/KIP-1356*3A*Introduce*IQv2*for*headers-aware*state*stores__;JSsrKysrKw!!Ayb5sqE7!oZFamFyZDrAmJeFHgKc-nOWUWL85bEbfHSdwgrp9jNHVOUCZgoOQ9aiK94L8zPr-tA_zx3WTmNs9vVDlqgQ$ > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> Thanks. > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>> > > >>>> > > >>> > > >> > > > > > > > >
