[
https://issues.apache.org/jira/browse/APEXCORE-577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897181#comment-15897181
]
Tushar Gosavi commented on APEXCORE-577:
----------------------------------------
Faced some issues while implementing suggested approach. The first entry point
to the user code
is populateDAG in the StreamingApplication. The plugins needs to be loaded
before prePopulateDAG
is called. We could go with ServiceLoader approach to load the plugin but it
has some issues, for example we can not have configuration for the plugin, as
ServiceLoader needs a default constructor to create an instance. And we do not
know from where to apply the plugin specific configuration before
calling plugins methods.
The preSetProperties/postSetProperty is tied up to the internal implementation
of DAG launching mechanism. We could have different implementation (python or
scala shell) which creates a DAG and submit for execution using launcher api
without setting configuration from xml files. for example a beam app launched
using maven cli, won't have preSetProperties/postSetProperty callbacks.
We have three implementation of preparing DAG as of now, all three needs to
call plugin specific functions in correct order before launch of DAG. Any new
mechanism to create DAG needs to remember to call plugin specific functions in
correct order.
I have my WIP changes in :
https://github.com/tushargosavi/apex-core/compare/master...tushargosavi:APEXCORE-577.visitor.new
Let me know your thoughts on this.
> Visitor API for DAG
> -------------------
>
> Key: APEXCORE-577
> URL: https://issues.apache.org/jira/browse/APEXCORE-577
> Project: Apache Apex Core
> Issue Type: Sub-task
> Reporter: Tushar Gosavi
> Assignee: Tushar Gosavi
>
> Support an Visitor like api for DAG, which could be used to write plugins for
> Apex.
> change DAG interface to add a visitor
> {code}
> registerVisitor(DagVisitor visitor)
> {code}
> The DagVisitor interface
> {code}
> interface DagVisitor {
> boolean startDAG(DAG* dag);
> boolean visitOperator(OperatorMeta *ometa)
> boolean visitStream(StreamMeta *smeta);
> boolean endDAG()
> }
> {code}
> dev thread discussions
> https://lists.apache.org/thread.html/1c37e0954cee029c9de537cc35ecb35beebdf49aba17de89bd2ce9ed@%3Cdev.apex.apache.org%3E
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
