Thanks for the proposal, I had a few comments on the APIs:
1. Printed#withFile seems not needed, as users should always spec if it is
to sysOut or to File at the beginning. In addition as a second thought, I
think serdes are not useful for prints anyways since we assume `toString`
is provided except for byte arrays, in which we will special handle it.
Another comment about Printed in general is it differs with other options
that it is a required option than optional one, since it includes toSysOut
/ toFile specs; what are the pros and cons for including these two in the
option and hence make it a required option than leaving them at the API
layer and make Printed as optional for mapper / label only?
2.1 KStream#through / to
We should have an overloaded function without Produced?
2.2 KStream#groupBy / groupByKey
We should have an overloaded function without Serialized?
2.3 KGroupedStream#count / reduce / aggregate
We should have an overloaded function without Materialized?
We should have an overloaded function without Joined?
2.5 Each of KTable's operators:
We should have an overloaded function without Produced / Serialized /
3.1 Produced: the static functions have overlaps, which seems not
necessary. I'd suggest jut having the following three static with another
three similar member functions:
public static <K, V> Produced<K, V> withKeySerde(final Serde<K> keySerde)
public static <K, V> Produced<K, V> withValueSerde(final Serde<V>
public static <K, V> Produced<K, V> withStreamPartitioner(final
StreamPartitioner<K, V> partitioner)
The key idea is that by using the same function name string for static
constructor and member functions, users do not need to remember what are
the differences but can call these functions with any ordering they want,
and later calls on the same spec will win over early calls.
3.2 Serialized: similarly
public static <K, V> Serialized<K, V> withKeySerde(final Serde<K> keySerde)
public static <K, V> Serialized<K, V> withValueSerde(final Serde<V>
public Serialized<K, V> withKeySerde(final Serde<K> keySerde)
public Serialized<K, V> withValueSerde(final Serde valueSerde)
Also it has a final Serde<V> otherValueSerde in one of its static
constructor, it that intentional?
3.3. Joined: similarly, keep the static constructor signatures the same as
its corresponding member fields.
3.4 Materialized: it is a bit special, and I think we can keep its static
constructors with only two `as` as they are today.K
4. Is there any modifications on StateStoreSupplier? Is it replaced by
BytesStoreSupplier? Seems some more descriptions are lacking here. Also in
public static <K, V, S extends StateStore> Materialized<K, V, S>
Is the parameter in type of BytesStoreSupplier?
On Thu, Jul 27, 2017 at 5:26 AM, Damian Guy <damian....@gmail.com> wrote:
> Updated link:
> On Thu, 27 Jul 2017 at 13:09 Damian Guy <damian....@gmail.com> wrote:
> > Hi,
> > I've put together a KIP to make some changes to the KafkaStreams DSL that
> > will hopefully allow us to:
> > 1) reduce the explosion of overloads
> > 2) add new features without having to continue adding more overloads
> > 3) provide simpler ways for people to use custom storage engines and wrap
> > them with logging, caching etc if desired
> > 4) enable per-operator caching rather than global caching without having
> > to resort to supplying a StateStoreSupplier when you just want to turn
> > caching off.
> > The KIP is here:
> > https://cwiki.apache.org/confluence/pages/viewpage.
> > Thanks,
> > Damian