As Tushar mentions, properties/attributes can be injected from external sources. We've already had multiple questions on the mailing list asking how to do this.
Ram On Thu, Nov 17, 2016 at 10:05 AM, Sanjay Pujare <[email protected]> wrote: > There is a risk if the user written code blocks the thread or crashes the > process. What are the real life examples of this use case? > > > On 11/17/16, 9:21 AM, "amol kekre" <[email protected]> wrote: > > +1. Opening up the API for users to put in their own code is good. In > general we should enable users to register their code in a lot of > scenerios. > > Thks > Amol > > On Thu, Nov 17, 2016 at 9:06 AM, Tushar Gosavi <[email protected] > > > wrote: > > > Yes, It could happen after current DAG validation and before the > > application master is launched. > > > > - Tushar. > > > > > > On Thu, Nov 17, 2016 at 8:32 PM, Munagala Ramanath < > [email protected]> > > wrote: > > > When would the visits happen ? Just before normal validation ? > > > > > > Ram > > > > > > On Wed, Nov 16, 2016 at 9:50 PM, Tushar Gosavi <[email protected]> > > wrote: > > > > > >> Hi All, > > >> > > >> How about adding visitor like API for DAG in Apex, and an api to > > >> register visitor for the DAG. > > >> Possible use cases are > > >> - Validator visitor which could validate the dag > > >> - Visitor to inject properties/attribute in the operator/streams > from > > >> some external sources. > > >> - Platform does not support validation of individual operators. > > >> developer could write a validator visitor which would call > validate > > >> function of operator if it implements Validator interface. > > >> - generate output schema based on operator config and input > schema, > > >> and set the schema on output stream. > > >> > > >> Sample API : > > >> > > >> dag.registerVisitor(DAGVisitor visitor); > > >> > > >> Call order of visitorFunctions. > > >> - preVisitDAG(Attributes) // dag attributes > > >> for all operators > > >> - visitOperator(OperatorMeta meta) // access to operator, name, > > >> attributes, properties > > >> ports > > >> - visitStream(StreamMeta meta) // access to > > >> stream/name/attributes/properties/ports > > >> - postVisitDAG() > > >> > > >> Regards, > > >> -Tushar. > > >> > > > > > >
