Hyunseok, >From the DAG it looks like single port. In case that is true, one way to get around this is to introduce a dummy pass-through operator during design of the app itself. This pass through operator shoudl have two input ports and one output port. One of the port is getting data from Y, other is open during launch time. During run time try inserting new operator P and connect it to this open port, and then disconned Y. A little tricky, but see if it works. The danger here is that the connects and disconnects should be triggered on the same window boundary, else you will have a window with both old and new operators are active.
Thks Amol On Tue, Aug 30, 2016 at 1:57 PM, Thomas Weise <[email protected]> wrote: > Hi, > > This depends on the operator Z. If it has multiple input ports and those > are optional, you can add P, then connect P to Z (and X), then remove Y. > > If Z has a single port, then Z and everything downstream would need to be > removed or else the change won't result in a valid DAG and won't be > accepted. > > Thomas > > > On Tue, Aug 30, 2016 at 1:48 PM, Hyunseok Chang <[email protected]> > wrote: > >> So if I replace Y in pipeline X -> Y -> Z -> U -> W -> V with P, then >> what I would have is X -> P -> Z' -> U' -> W' -> V' ? Where Z', U', W' >> and V' are new operator instances that need to be deployed along with P. >> >> Is my understanding correct? If so is there any reason why we cannot >> re-use existing operators downstream? >> >> -hs >> >> >> On Tue, Aug 30, 2016 at 2:46 PM, Amol Kekre <[email protected]> wrote: >> >>> >>> Hyunseok, >>> The new route in the pipeline will have a new Z operator. If you want to >>> use the old Z operator (state?) then things get tricky. Do confirm that you >>> do not plan to use old Z operator. >>> >>> Thks, >>> Amol >>> >>> >>> On Tue, Aug 30, 2016 at 11:02 AM, Sandesh Hegde <[email protected] >>> > wrote: >>> >>>> Hello hs, >>>> >>>> Yes, you can change the topology from the Apex CLI. >>>> >>>> One possible sequence of commands for your scenario is described below, >>>> >>>> connect appid >>>> begin-logical-plan-change >>>> create-operator .... >>>> add-stream-sink ... ( for the input of P ) >>>> add-stream-sink ... ( for the output of P ) >>>> remove-operator ... >>>> submit >>>> >>>> Note: All the required operators needs to be in the package. >>>> >>>> Thanks >>>> >>>> >>>> >>>> >>>> >>>> On Tue, Aug 30, 2016 at 7:22 AM Hyunseok Chang < >>>> [email protected]> wrote: >>>> >>>>> Hi, >>>>> >>>>> I'd like to know more about Apex support for dynamic topology. >>>>> >>>>> From my readings on Apex, I understand we can add additional parallel >>>>> tasks for each operator and change data partitioning among them >>>>> dynamically >>>>> at run time (so-called data partitioning and unification features). >>>>> >>>>> My question is can we change the "logical" DAG at run time? >>>>> >>>>> Let's say my logical DAG is a chain of three operators X, Y & Z (i.e., >>>>> X -> Y -> Z). Now at run time I want to replace operator Y with operator >>>>> P, such that the new logical DAG would look like X -> P -> Z. >>>>> >>>>> Is it something I can do with Apex? >>>>> >>>>> Thanks! >>>>> -hs >>>>> >>>>> >>> >> >
