Re: [Proposal] DAG listener

2016-08-15 Thread Sandesh Hegde
@Sanjay Here is the issue I mentioned https://issues.apache.org/jira/browse/APEXCORE-496 Instead of exposing one information at a time, why not publish all the information? To start with it could just be Physical DAG and update the subscribers after partitioning. Thanks On Wed, Aug 10, 2016 at

Re: [Proposal] DAG listener

2016-08-10 Thread Sandesh Hegde
Any operators can subscribe to Stram Events affecting the DAG. Implementation will most probably use heartbeat. On Tue, Aug 9, 2016 at 11:43 PM Tushar Gosavi wrote: > Hi Sandesh, > > Dag changes are handled in Stram, and operators are running in > different containers.

Re: [Proposal] DAG listener

2016-08-10 Thread Tushar Gosavi
Hi Sandesh, Dag changes are handled in Stram, and operators are running in different containers. Are you suggesting an RPC interface or operator request for sending this information from Stram to all partitions of the interested operator? - Tushar. On Wed, Aug 10, 2016 at 11:28 AM, Sandesh

[Proposal] DAG listener

2016-08-09 Thread Sandesh Hegde
Hi All, As we add more features to support batch use cases, there will be a need to access more information about the DAG from an operator. One example is the need to access the operator name while using Batched StatsListener. The idea here is to implement DAG Listener ( similar to StatsListener