Re: KIP-213- [DISCUSS] - Three follow-up discussion points - topic partitioning, serializers, hashers

2019-03-19 Thread Adam Bellemare
Thanks John & Matthias. I have created a report with Confluent ( https://github.com/confluentinc/schema-registry/issues/1061). I will continue on with current work and we can resume the discussion, as Matthias correctly indicates, in the PR. Matthias, thank you for the link to Kafka-. This is

Re: KIP-213- [DISCUSS] - Three follow-up discussion points - topic partitioning, serializers, hashers

2019-03-19 Thread John Roesler
Chiming in... 1) Agreed. There is a technical reason 1:1 joins have to be co-partitioned, which does not apply to the many:1 join you've designed. 2) Looking at the Serializer interface, it unfortunately doesn't indicate whether the topic (or the value) is nullable. There are several places in

Re: KIP-213- [DISCUSS] - Three follow-up discussion points - topic partitioning, serializers, hashers

2019-03-18 Thread Matthias J. Sax
Just my 2 cents. Not sure if others see it differently: 1) it seems that we can lift the restriction on having the same number of input topic partitions, and thus we should exploit this IMHO; don't see why we should enforce an artificial restriction 2) for the value serde it's a little bit more

KIP-213- [DISCUSS] - Three follow-up discussion points - topic partitioning, serializers, hashers

2019-03-17 Thread Adam Bellemare
Hey folks I have been implementing the KIP as outlined in https://cwiki.apache.org/confluence/display/KAFKA/KIP-213+Support+non-key+joining+in+KTable, and I have run into a few points to consider that we did not include in the original. *1) Do all input topics need to have the same partitions or