Hi all, While working on JStorm and MapReduce runners, I found that it is very helpful to understand Beam pipelines by visualizing them.
Logical graph: https://drive.google.com/file/d/0B6iZ7iRh-LOYc0dUS0Rwb2tvWGM/view?usp=sharing Physical graph: https://drive.google.com/file/d/0B6iZ7iRh-LOYbDFWeDlCcDhnQmc/view?usp=sharing I think we can visualize Beam logical DAG in runner-core. It should also be easy to visualize the physical DAG in each runners. (Maybe we can define some shared data structures to make it more automatic, and even support visualizing them in Apex/Flink/Spark/Gearpump UIs). I have a commit for MapReduce runner in here (<200 lines). And, this commit generates dotfiles for logical and physical DAGs. https://github.com/peihe/incubator-beam/commit/bb3349e10c0cfacd81b610880ddfec030fedf34d Looking forward to ideas and feedbacks. -- Pei
