Hello Paul,
Thanks for the great writeup (very detailed and crystal motivation
sections!).
This is quite an interesting idea and I do like the API cleanness you
proposed. The original motivation of letting StreamsTopology to add state
stores though, is to allow different processors to share the state store.
For example:
builder.addStore("store1");
// a path of stream transformations that leads to KStream stream1.
stream1.transform(..., "store1");
// another path that generates a KStream stream2.
stream2.transform(..., "store1");
Behind the scene, Streams will make sure stream1 / stream2 transformations
will always be grouped together as a single group of tasks, each of which
will be executed by a single thread and hence there's no concurrency issues
on accessing the store from different operators within the same task. I'm
not sure how common this use case is, but I'd like to hear if you have any
thoughts maintaining this since the current proposal seems exclude this
possibility.
Guozhang
On Sun, Dec 9, 2018 at 4:18 PM Paul Whalen <pgwha...@gmail.com> wrote:
> Here's KIP-401 for discussion, a minor Kafka Streams API change that I
> think could greatly increase the usability of the low-level processor API.
> I have some code written but will wait to see if there is buy in before
> going all out and creating a pull request. It seems like most of the work
> would be in updating documentation and tests.
>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=97553756
>
> Thanks!
> Paul
>
--
-- Guozhang
