Hey everyone,
Two more minor updates to the KIP came out of wrapping up the
implementation and discussions with Guozhang and Matthias offline:
- Versioned stores will be disabled for global tables and also the
suppress operator, in order to limit the scope of these changes and to
Thanks for your comments and suggestions, Matthias, Lucas, and Guozhang!
I was just in the process of responding when I saw Guozhang's message. I
came up with a different approach to simplify my proposal with regards to
the table aggregate processor, as part of mulling over comments from
Matthias
Thanks Victoria.
1) I have no concerns about the filter operator's proposed semantics.
2) For aggregations, I have meta question in mind to discuss first,
which is for those operators that generate a table, which is
materialized as versioned, how we should emit out of order data if the
operator
If we send old and new value as two messages, this should work I guess?
Victory could confirm. -- But not if we send old/new as a single message
in case the new-key does not change?
-Matthias
On 4/11/23 5:25 AM, Lucas Brutschy wrote:
Hi,
No concerns at all, just a clarifying question from
Hi,
No concerns at all, just a clarifying question from my side: for
detecting out-of-order records, I need both new and old timestamp, I
suppose I get it for the new record via timestamp extractor, can I not
get it the same way from the old record that is passed down to the
aggregation after
Thanks.
One question: for the repartition topic format change, do we want to
re-use flag=2, or should we introduce flag=3, and determine when
compiling the DSL into the Topology if we want/need to include the
timestamp, and if not, use format version=2 to avoid unnecessary overhead?
Hi everyone,
While wrapping up the implementation for KIP-914, I have discovered that
two more DSL processors require semantic updates in the presence of
versioned tables:
- The table filter processor has an optimization to drop nulls if the
previous filtered value is also null. When the
