As a small follow up, here are parts of the code that I stumbled upon. Each edge that has a vertex with 0 tasks as its destination, says that no routing of events is required: https://github.com/apache/tez/blob/4b9a7be1b98cff00c44e7d3ffb2486bb59ca6804/tez-dag/src/main/java/org/apache/tez/dag/app/dag/impl/Edge.java#L263-L264
That causes the EdgeManager code to be skipped: https://github.com/apache/tez/blob/4b9a7be1b98cff00c44e7d3ffb2486bb59ca6804/tez-dag/src/main/java/org/apache/tez/dag/app/dag/impl/Edge.java#L487-L527 Is there another mechanism that the VertexManager uses to take action on data movement events, other than routing through the edge manager? > -----Original Message----- > From: Adrian Nicoara <[email protected]> > Sent: Monday, August 13, 2018 1:43 PM > To: [email protected]; Hitesh Sharma <[email protected]>; . Anupam > <[email protected]> > Subject: RE: TEZ-1190 status > > > -----Original Message----- > > From: Gopal Vijayaraghavan <[email protected]> > > Sent: Friday, August 10, 2018 2:20 PM > > To: [email protected]; Hitesh Sharma <[email protected]>; . > Anupam > > <[email protected]> > > Subject: Re: TEZ-1190 status > > > > As far as I know, that patch + design has been abandoned. > > > > There was a discussion about adding it without fundamentally changing > > the DAGImpl (and task recovery etc). > > > > The VertexManager already allows you to have a vertex without a task > > in it, without giving up on being a participant in the data movement events. > > This sounds very promising. Can you point me to any tests, or sample code, > or the implementation code I should be looking at? > > > However, there are no immediate takers for the new approach (which is > > additive, so needs no significant refactoring changes) and this > > feature did not have anyone waiting for it. > > If it can be done with the above suggestion, I can pick up this work, if it > is OK.
