Hi, I have read from the documentation that if you have more spout tasks than kafka partition, the excessive tasks will remain idle for entire lifecycle of the topology.
Now, Let's consider 4 spout tasks, 32 bolt tasks (of one class) in 4 workers (in 4 nodes) and 2 partitions in kafka. Then 2 tasks will be assigned to each partitions in kafka and the other 2 will remain idle. However, does that mean that only the bolts within the same worker will get the messages (assuming shuffle grouping)? Or, do the messages get emitted to whatever bolt taks available, regardless of which worker? Thanks, Baek *Seungtack Baek | Precocity, LLC* Tel/Direct: (972) 378-1030 | Mobile: (214) 477-5715 *[email protected] <[email protected]>* | www.precocityllc.com This is the end of this message. -- On Sun, Jun 7, 2015 at 10:12 PM, Seungtack Baek < [email protected]> wrote: > Hi, > > I have read from the documentation that if you have more spout tasks than > kafka partition, the excessive tasks will remain idle for entire lifecycle > of the topology. > > Now, Let's consider 4 spout tasks, 32 bolt tasks (of one class) in 4 > workers (in 4 nodes) and 2 partitions in kafka. Then 2 tasks will be > assigned to each partitions in kafka and the other 2 will remain idle. > However, does that mean that only the bolts within the same worker will get > the messages (assuming shuffle grouping)? Or, do the messages get emitted > to whatever bolt taks available, regardless of which worker? > > Thanks, > Baek > > > *Seungtack Baek | Precocity, LLC* > > Tel/Direct: (972) 378-1030 | Mobile: (214) 477-5715 > > *[email protected] <[email protected]>* | > www.precocityllc.com > > > This is the end of this message. > > -- >
