Yes Amol, the PR that Thomas pointed out is for that purpose. It allows to change the plan at runtime. We can remove all opertors entirely and add new operators to the existing dag.
~ Bhupesh On Jan 16, 2017 23:38, "Amol Kekre" <a...@datatorrent.com> wrote: > Bhupesh, > If stram changes the logical dag (runs only the sub-dag) on each runs, it > should be ok. > > Thks > Amol > > > On Mon, Jan 16, 2017 at 9:18 AM, Bhupesh Chawda <bhup...@datatorrent.com> > wrote: > > > Yes, I thought of that. That way it will be a single application > > throughout. > > Just wanted to see if there can be other options. > > > > ~ Bhupesh > > > > On Mon, Jan 16, 2017 at 9:17 PM, Thomas Weise <t...@apache.org> wrote: > > > > > Sounds like a fit for https://github.com/apache/apex-core/pull/410 ? > > > > > > On Mon, Jan 16, 2017 at 3:27 AM, Bhupesh Chawda < > bhup...@datatorrent.com > > > > > > wrote: > > > > > > > Hi All, > > > > > > > > We have a use case where I need to launch a number of DAGs on the > > cluster > > > > one after the other in sequence programatically. > > > > > > > > We are using the StramAppLauncher and StramAppFactory classes to > > launch a > > > > DAG programatically on the cluster and adding any third party > > > dependencies > > > > as part of the configuration. > > > > > > > > It is working fine except for the following issue: > > > > Every time a DAG is launched, it copies the dependencies to the > > > application > > > > folder and hence spends a good amount of time before the app actually > > > > starts running. All of the apps I run belong to the same project and > > > hence > > > > don't actually need separate set of jars. > > > > > > > > Is there any way I can make all the applications "share" the jars > which > > > are > > > > uploaded when the first application is run? > > > > > > > > Thanks. > > > > > > > > ~ Bhupesh > > > > > > > > > >
