The main purpose is not to handle back pressure but to limit bandwidth usage by applications. This is useful in ingestion use cases. Typically user needs to ingest say up to 1GB per sec and not more. The tuple size may vary based on messages based tuples (few KBs) or block tuples for files (few MBs). Bandwidth manager will take max bandwidth that can be utilized by the application and will take care of sharing that across partitions etc.
Priyanka: You could also consider following in your design 1. Limiting input rate (across partitions) 2. Limiting output rate (across partitions) 3. Specifying total bandwidth that the Application can utilize including input and output? Not sure if this is required. Need comments from others here. 4. Include default implementation that will handle 1 and 2, and anyone interested in having their own Bandwidth manager should be able to extend the default one. 5. Can you also look at including/extending tuples per sec as pointed out by Tim/Chinmay. Regards, Sandeep On Thu, Mar 3, 2016 at 12:23 AM, Timothy Farkas <[email protected]> wrote: > Not sure if this is helpful, but there is already a utility in Malhar for > converting tuples per second to tuples per window. This allows the user to > define a property in tuples per second, then the operator can convert that > to tuples per window so it emits the correct number of tuples per window. > > > https://github.com/apache/incubator-apex-malhar/blob/master/library/src/main/java/com/datatorrent/lib/util/time/WindowUtils.java > > On Wed, Mar 2, 2016 at 10:41 AM, Chinmay Kolhatkar < > [email protected]> > wrote: > > > Hi Priyanka, > > > > Indeed this is a useful feature. > > > > I believe number bytes consumed per sec can as well translate to number > of > > tuples consumed per sec. > > > > If above is correct, won't back pressure that is handled by bufferserver > > help in your use case? > > > > Thanks, > > Chinmay. > > On 2 Mar 2016 4:49 p.m., "Priyanka Gugale" <[email protected]> > > wrote: > > > > > Many times we need to put bandwidth restrictions or put some limit on > > input > > > operator for number of bytes to be consumed per second. As I understand > > in > > > Apex there is no direct support for this feature. > > > > > > I am planning to write a bandwidth manager which will help in limiting > > > bandwidth at Input operator. Let me know if there are any better > > > alternative ways. I will soon publish design for Bandwidth Manager I am > > > planning to write. > > > > > > -Priyanka > > > > > >
