Hey Erik Thanks for reply ,
Actually, I am working on custom scheduler for storm where we can explicitly specify where to place the thread/executor to particular slot mapping on a VM . In order to improve N/W communication cost btw threads .In some papers, I found like if DAG is - [s0------> b1--------->b2------------>b3] , where every bolt have some parallelism (say 3 for all ) . connected by shuffle grouping and placement is done say . *[VM1(s0------> b1--------->b2------------>b3),VM2(s0------> b1--------->b2------------>b3),VM3(s0------> b1--------->b2------------>b3)] (1 thread of each one on diff VMs)* then the instance of spout *s0 on VM1* will also be sending some tuples to b1 bolt instance on other VMs .(as per definition of shuffle grouping) . On Sun, Apr 10, 2016 at 3:03 PM, Erik Weathers <[email protected]> wrote: > hi Anshu. Can you please clarify what is "not clear" to you in the > description? > > On Sun, Apr 10, 2016 at 2:31 AM, anshu shukla <[email protected]> > wrote: > >> Thanks Xiang, >> >> I have read it already . But I am not clear about this point - >> >> >> 1. *Local or shuffle grouping: If the target bolt has one or more >> tasks in the same worker process, tuples will be shuffled to just those >> in-process tasks. Otherwise, this acts like a normal shuffle grouping* >> >> >> On Sun, Apr 10, 2016 at 6:55 AM, Xiang Wang <[email protected]> wrote: >> >>> Hi, >>> >>> Please have a look at this page: >>> http://storm.apache.org/releases/0.10.0/Concepts.html >>> >>> There are many different grouping methods, and direct grouping may be >>> what you are looking for. >>> >>> >>> ------------------------------- >>> Xiang Wang PhD Candidate >>> Database Research Group >>> School of Computer Science and Engineering >>> The University of New South Wales >>> Sydney, Australia >>> >>> On Sun, Apr 10, 2016 at 7:14 AM, anshu shukla <[email protected]> >>> wrote: >>> >>>> Suppose we have 10 exec/threads of spouts sending tuples to 5 threads >>>> of downstream bolt (say B) .Then my question is - >>>> >>>> *Does every thread of spout will uniformly distribute messages to all >>>> the 5 threads of downstream nodes .* >>>> >>>> *Or * >>>> >>>> *Is there any one to one communication btw spouts exec to bolts .* >>>> >>>> >>>> Very relevant if we want to improve N/W cost by custom scheduler in >>>> storm. >>>> >>>> -- >>>> Thanks & Regards, >>>> Anshu Shukla >>>> >>> >>> >> >> >> -- >> Thanks & Regards, >> Anshu Shukla >> > > -- Thanks & Regards, Anshu Shukla
