@Sanjay Here is the issue I mentioned https://issues.apache.org/jira/browse/APEXCORE-496
Instead of exposing one information at a time, why not publish all the information? To start with it could just be Physical DAG and update the subscribers after partitioning. Thanks On Wed, Aug 10, 2016 at 10:40 AM Sanjay Pujare <[email protected]> wrote: > Looks like a good idea but would like more details about the use case > (“example ... to access the operator name while using Batched > StatsListener”). > > Do you propose very fine grained subscription model? Also what “major > activities in the DAG” will be available thru this listener? > > > > On 8/10/16, 8:20 AM, "Sandesh Hegde" <[email protected]> wrote: > > Any operators can subscribe to Stram Events affecting the DAG. > > Implementation will most probably use heartbeat. > > On Tue, Aug 9, 2016 at 11:43 PM Tushar Gosavi <[email protected]> > wrote: > > > Hi Sandesh, > > > > Dag changes are handled in Stram, and operators are running in > > different containers. > > Are you suggesting an RPC interface or operator request for sending > > this information > > from Stram to all partitions of the interested operator? > > > > - Tushar. > > > > > > > > On Wed, Aug 10, 2016 at 11:28 AM, Sandesh Hegde < > [email protected]> > > wrote: > > > Hi All, > > > > > > As we add more features to support batch use cases, there will be > a need > > to > > > access more information about the DAG from an operator. One > example is > > the need > > > to access the operator name while using Batched StatsListener. > > > > > > The idea here is to implement DAG Listener ( similar to > StatsListener ), > > > which can potentially give every information present in the Stram. > > > Operators will also have visibility into all the major activities > in the > > > DAG. > > > > > > Thoughts? > > > > > > Thanks > > > > > >
