Good question. All of them would run in a single thread. That is the model. Multiple threads would make sense to run separate DAGs.
Eno > On 25 Aug 2016, at 18:32, Abhishek Agarwal <abhishc...@gmail.com> wrote: > > Hi Eno, > > Thanks for your reply. If my application DAG has three stream processors, > first of which is source, would all of them run in single thread? There may > be scenarios wherein I want to have different number of threads for > different processors since some may be CPU bound and some may be IO bound. > > On Thu, Aug 25, 2016 at 10:49 PM, Eno Thereska <eno.there...@gmail.com> > wrote: > >> Hi Abhishek, >> >> - Correct on connecting to external stores. You can use Kafka Connect to >> get things in or out. (Note that in the 0.10.1 release KIP-67 allows you to >> directly query Kafka Stream's stores so, for some kind of data you don't >> need to move it to an external store. This is pushed in trunk.) >> >> - You can definitely use more threads than partitions, but that will not >> buy you much since some threads will be idle. No two threads will work on >> the same partition, so you don't have to worry about them repeating work. >> >> Hope this helps. >> Eno >> >>> On 25 Aug 2016, at 16:50, Abhishek Agarwal <abhishc...@gmail.com> wrote: >>> >>> Hi, >>> I was reading up on kafka streams for a project and came across this blog >>> https://softwaremill.com/kafka-streams-how-does-it-fit-stream-landscape/ >>> I wanted to validate some assertions made in blog, with kafka community >>> >>> - Kafka streams is kafka-in, kafka-out application. Does the user need >>> kafka connect to transfer data from kafka to any external store? >>> - No support for asynchronous processing - Can I use more threads than >>> number of partitions for processors without sacrificing at-least once >>> guarantees? >>> >>> >>> >>> -- >>> Regards, >>> Abhishek Agarwal >> >> > > > -- > Regards, > Abhishek Agarwal
