Thanks Chetan, A followup question to this is let's say Operator A is partitioned, and each of the partitions is in a different window. When a property is set dynamically on A will it be set on all the partitions within the same window?
Thanks, Tim On Mon, Sep 21, 2015 at 10:12 AM, Chetan Narsude <[email protected]> wrote: > Got it. Thanks for the explanation. This operation is in the same class as > the class for "Join" operation. The simplest we have came across is > division operation. Where you need both numerator and denominator to do the > division. And there are a few complicated ones that we have solved in the > past. > > Since this is application logic - tell me why the following may not work: > > 1. Have 2 output ports for A. 1st for schema and the 2nd for events. > 2. Have 2 input ports for B. 1st receives schema and the 2nd one receives > events. > 3. If an event is received before the schema - buffer it till the schema > shows up > 4. If the schema shows up before the events - save the schema for later > use. > > -- > Chetan > > On Mon, Sep 21, 2015 at 10:59 AM, Timothy Farkas <[email protected]> > wrote: > > > I think this is non-abstract enough, if you need more detail we can talk > > offline. The use case is that I have two operators Operators A and B. > > Operator A processes data according to a schema and emits tuples > > corresponding to that schema which is tagged with a schema ID. Operator B > > receives the processed data checks the schema ID and looks up the > > appropriate meta information to continue processing the tuples. If I want > > to make a change to the schema that is used to process the tuples while > the > > application is running I need to set a property which updates the schema > > for both Operators A and B. If the schema is updated for Operator A > before > > it is updated for Operator B, that is bad because then Operator B will be > > receiving data it doesn't understand yet. If the schema is updated on > > Operator B and then Operator A that is okay, but it requires the user to > be > > careful about monitoring their properties and introduces the possibility > of > > user error. > > > > Thanks, > > Tim > > > > On Mon, Sep 21, 2015 at 9:35 AM, Chetan Narsude <[email protected]> > > wrote: > > > > > Tim, can you share the specific (non-abstract) use case if appropriate > > > publicly? Or PM me. > > > > > > -- > > > Chetan > > > > > > On Mon, Sep 21, 2015 at 10:30 AM, David Yan <[email protected]> > > wrote: > > > > > > > Currently setting of operator properties are done by issuing > > > > StramToNodeRequests, which means that it will only be handled at > window > > > > boundaries (specifically at END_WINDOW) upon receipt of the heartbeat > > > > response from STRAM. > > > > > > > > I agree with Amol. Your requirement will need such operations to be > > > > injected in a control tuple. > > > > > > > > David > > > > > > > > On Sun, Sep 20, 2015 at 12:30 AM, Amol Kekre <[email protected]> > > > wrote: > > > > > > > > > Tim, > > > > > We will need to add a new control tuple. The only way to set > > properties > > > > on > > > > > two operators at the begining of the same window is via a control > > > tuple. > > > > > Other way may be that Stram sends information to StramChild with a > > > window > > > > > ID way in the future. Second approach is not good, and > theoratically > > > does > > > > > not guarantee the alignment anyway. > > > > > > > > > > Thks, > > > > > Amol > > > > > > > > > > > > > > > On Sat, Sep 19, 2015 at 2:58 PM, Timothy Farkas < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > Hello all, > > > > > > > > > > > > If I dynamically set a property on an operator, does the value of > > > that > > > > > > property get updated between windows or could it get updated in > the > > > > > middle > > > > > > of a window? Also is it possible to set multiple properties on > > > > operators > > > > > > simultaneously? > > > > > > > > > > > > For example let's say I have Operator A with Property 1 and > > Operator > > > B > > > > > with > > > > > > Property 2. If I want to set both Properties 1 and 2 on > Operators A > > > > and B > > > > > > right before window N in both operators, is there a mechanism by > > > which > > > > I > > > > > > can do that? > > > > > > > > > > > > I think this could be accomplished by creating a separate update > > > > operator > > > > > > on which properties are set, and which emits the updates to > > > downstream > > > > > > operators to be updated, but I am wondering if there is a better > > > > > solution. > > > > > > > > > > > > Thanks, > > > > > > Tim > > > > > > > > > > > > > > > > > > > > >
