Thanks Marco, Sorry if I caused any trouble!
I don’t remember what I was thinking before, but reasoning about it now, you might need the fine-grained choice if: 1. The number or size of records in each partition of both tables is small(ish), but the cardinality of the join is very high. Then you might want an in-memory table store, but an on-disk subscription store. 2. The number or size of records is very large, but the join cardinality is low. Then you might need an on-disk table store, but an in-memory subscription store. 3. You might want a different kind (or differently configured) store for the subscription store, since it’s access pattern is so different. If you buy these, it might be good to put the justification into the KIP. I’m in favor of the default you’ve proposed. Thanks, John On Mon, Mar 29, 2021, at 04:24, Marco Aurélio Lotz wrote: > Hi Guozhang, > > Apologies for the late answer. Originally that was my proposal - to > piggyback on the provided materialisation method ( > https://issues.apache.org/jira/browse/KAFKA-10383). > John Roesler suggested to us to provide even further fine tuning on API > level parameters. Maybe we could see this as two sides of the same coin: > > - On the current API, we change it to piggy back on the materialization > method provided to the join store. > - We extend the API to allow a user to fine tune different materialization > methods for subscription and join store. > > What do you think? > > Cheers, > Marco > > On Thu, Mar 4, 2021 at 8:04 PM Guozhang Wang <wangg...@gmail.com> wrote: > > > Thanks Marco, > > > > Just a quick thought: what if we reuse the existing Materialized object for > > both subscription and join stores, instead of introducing a new param / > > class? > > > > Guozhang > > > > On Tue, Mar 2, 2021 at 1:07 AM Marco Aurélio Lotz <cont...@marcolotz.com> > > wrote: > > > > > Hi folks, > > > > > > I would like to invite everyone to discuss further KIP-718: > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-718%3A+Make+KTable+Join+on+Foreign+key+unopinionated > > > > > > I welcome all feedback on it. > > > > > > Kind Regards, > > > Marco Lotz > > > > > > > > > -- > > -- Guozhang > > >
