Could you have a combined filterAB that adds a new field, "which" populated with "a" or "b", and then partition the stream based on that field?
--Tom On Thursday, September 24, 2015, Grant Overby (groverby) <[email protected]> wrote: > Yes > > *Grant Overby* > Software Engineer > Cisco.com <http://www.cisco.com/> > [email protected] <javascript:_e(%7B%7D,'cvml','[email protected]');> > Mobile: *865 724 4910* > > > > Think before you print. > > This email may contain confidential and privileged material for the sole > use of the intended recipient. Any review, use, distribution or disclosure > by others is strictly prohibited. If you are not the intended recipient (or > authorized to receive for the recipient), please contact the sender by > reply email and delete all copies of this message. > > Please click here > <http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for > Company Registration Information. > > > > > From: Tom Brown <[email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');>> > Reply-To: "[email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');>" < > [email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');>> > Date: Thursday, September 24, 2015 at 1:04 PM > To: "[email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');>" < > [email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');>> > Subject: Re: Tident: fork DAG without partitioning? > > Are your filters such that a tuple will pass either filterA, or filterB, > but never both? > > --Tom > > On Thu, Sep 24, 2015 at 10:35 AM, Grant Overby (groverby) < > [email protected] <javascript:_e(%7B%7D,'cvml','[email protected]');>> > wrote: > >> I have a trident topology where a portion of the DAG looks like this: >> >> >> partition - - - filterA - - - eachA >> / >> stream - - - - >> \ >> partition - - - filterB - - - eachB >> >> >> >> I believe with the above DAG, each tuple will be sent down both sides of >> the fork. Approximately half will be filtered out by each filter. Thus, >> forcing half my tuples to cross the partition only to get dropped >> afterwards. >> >> Can the DAG be constructed like the following. If so, how do I define the >> topology? >> >> filterA - - - partition - - - eachA >> / >> stream - - - - >> \ >> filterB - - - partition - - - eachB >> >> >> >> >> The following doesn’t appear to work. filterA and filterB don’t receive >> tuples. >> >> Stream stream = …; >> >> stream.parallelismHint(N); >> >> stream >> .each(new Fields(…), filterA) >> .localOrShuffle() >> .each(new Fields(…), eachA, new Fields(…)) >> ; >> >> stream >> .each(new Fields(…), filterB) >> .localOrShuffle() >> .each(new Fields(…), eachB, new Fields(…)) >> ; >> >> >> >> >> >> >> *Grant Overby* >> Software Engineer >> Cisco.com <http://www.cisco.com/> >> [email protected] <javascript:_e(%7B%7D,'cvml','[email protected]');> >> Mobile: *865 724 4910 <865%20724%204910>* >> >> >> >> Think before you print. >> >> This email may contain confidential and privileged material for the sole >> use of the intended recipient. Any review, use, distribution or disclosure >> by others is strictly prohibited. If you are not the intended recipient (or >> authorized to receive for the recipient), please contact the sender by >> reply email and delete all copies of this message. >> >> Please click here >> <http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for >> Company Registration Information. >> >> >> >> >
