Author: tzolov
Date: Sat Jan 10 01:40:25 2015
New Revision: 1650710
URL: http://svn.apache.org/r1650710
Log:
CRUNCH-487: Document the pipeline execution plan visualization configuration.
Modified:
crunch/site/trunk/content/user-guide.mdtext
Modified: crunch/site/trunk/content/user-guide.mdtext
URL:
http://svn.apache.org/viewvc/crunch/site/trunk/content/user-guide.mdtext?rev=1650710&r1=1650709&r2=1650710&view=diff
==============================================================================
--- crunch/site/trunk/content/user-guide.mdtext (original)
+++ crunch/site/trunk/content/user-guide.mdtext Sat Jan 10 01:40:25 2015
@@ -1602,3 +1602,20 @@ By embedding our business logic inside o
computations that combine custom DoFns with Crunch's built-in `cogroup`
operation by using the [MemPipeline](#mempipeline)
implementation to create test data sets that we can easily verify by hand, and
then this same logic can be executed on
a distributed data set using either the [MRPipeline](#mrpipeline) or
[SparkPipeline](#sparkpipeline) implementations.
+
+### Pipeline execution plan visualizations
+
+Crunch provides tools to visualize the pipeline execution plans. The
[PipelineExecution](apidocs/0.10.0/org/apache/crunch/PipelineExecution.html)
+`String getPlanDotFile()` method returns an execution plan visualization in
DOT format. If the dot file output folder property is set, Crunch produces a
DOT file after each pipeline run.
+
+Additional aspects of the execution plans are provided when the DOT file debug
mode is enabled. Then Crunch provides 4 additional DOT diagrams visualizing
different internal stages of the execution plan. Such plans include PCollection
lineage, Base graph plan, Split graph plans, Run-time nodes.
+Note: To enable the debug mode you should set an out put folder first. The
following snapped switches the DOT file debug mode. As a result 5 DOT diagrams
are generated in the output folder after each Pipeline execution:
+
+ Configuration conf = ...
+ String dotfileDir = ...
+
+ DotfileUtills.setPipelineDotfileOutputDir(conf, dotfileDir);
+ DotfileUtills.enableDebugDotfiles(conf);
+
+
+
