Hi guys, During some of my experiments, I realized that a vertex which is managed by a ShuffleVertexManager is looking for tasks who have finished just in parent vertices where data movement is SCATTER_GATHER.
For example, in the attached DAG, Reducer 3 is able to start tasks looking just at Map_5 and Reducer_2, and such even if none of the tasks have finished on the branch with Reducer 8, Reducer 3 still starts. The main reason seems to be that a ShuffleVertexManager is looking for tasks finished just in bipartiteSources vertices, which seems to be only those which are SCATTER_GATHER. So a parent which is Broadcast, is completely ignored from this. Do I miss something, i.e. it is as designed or it is a bug ? Thanks, Robert
