Yes, It could happen after current DAG validation and before the application master is launched.
- Tushar. On Thu, Nov 17, 2016 at 8:32 PM, Munagala Ramanath <[email protected]> wrote: > When would the visits happen ? Just before normal validation ? > > Ram > > On Wed, Nov 16, 2016 at 9:50 PM, Tushar Gosavi <[email protected]> wrote: > >> Hi All, >> >> How about adding visitor like API for DAG in Apex, and an api to >> register visitor for the DAG. >> Possible use cases are >> - Validator visitor which could validate the dag >> - Visitor to inject properties/attribute in the operator/streams from >> some external sources. >> - Platform does not support validation of individual operators. >> developer could write a validator visitor which would call validate >> function of operator if it implements Validator interface. >> - generate output schema based on operator config and input schema, >> and set the schema on output stream. >> >> Sample API : >> >> dag.registerVisitor(DAGVisitor visitor); >> >> Call order of visitorFunctions. >> - preVisitDAG(Attributes) // dag attributes >> for all operators >> - visitOperator(OperatorMeta meta) // access to operator, name, >> attributes, properties >> ports >> - visitStream(StreamMeta meta) // access to >> stream/name/attributes/properties/ports >> - postVisitDAG() >> >> Regards, >> -Tushar. >>
