Author: tzolov
Date: Sun Jan 11 17:27:41 2015
New Revision: 1650928
URL: http://svn.apache.org/r1650928
Log:
edit the pipeline visualization section
Modified:
crunch/site/trunk/content/user-guide.mdtext
Modified: crunch/site/trunk/content/user-guide.mdtext
URL:
http://svn.apache.org/viewvc/crunch/site/trunk/content/user-guide.mdtext?rev=1650928&r1=1650927&r2=1650928&view=diff
==============================================================================
--- crunch/site/trunk/content/user-guide.mdtext (original)
+++ crunch/site/trunk/content/user-guide.mdtext Sun Jan 11 17:27:41 2015
@@ -1605,16 +1605,20 @@ a distributed data set using either the
### Pipeline execution plan visualizations
-Crunch provides tools to visualize the pipeline execution plans. The
[PipelineExecution](apidocs/0.10.0/org/apache/crunch/PipelineExecution.html)
`String getPlanDotFile()` method returns an execution plan visualization in DOT
format. When the output folder property is set, Crunch produces a DOT file
after each pipeline run.
+Crunch provides tools to visualize the pipeline execution plan. The
[PipelineExecution](apidocs/0.10.0/org/apache/crunch/PipelineExecution.html)
`String getPlanDotFile()` method returns a DOT format visualization of the
exaction plan. Furthermore if the output folder is set then Crunch will save
the dotfile diagram on each pipeline execution:
-Additional aspects of the execution plans are provided when the DOT file debug
mode is enabled. Then Crunch provides 4 additional DOT diagrams visualizing
different internal stages of the execution plan. Such plans include PCollection
lineage, Base graph plan, Split graph plans, Run-time nodes.
-Note: To enable the debug mode you should set an out put folder first. The
following snapped switches the DOT file debug mode. As a result 5 DOT diagrams
are generated in the output folder after each Pipeline execution:
-
- Configuration conf = ...
- String dotfileDir = ...
-
+ Configuration conf =...;
+ String dotfileDir =...;
+
+ // Set DOT files out put folder path
DotfileUtills.setPipelineDotfileOutputDir(conf, dotfileDir);
- DotfileUtills.enableDebugDotfiles(conf);
+Additional details of the Crunch execution plan can be exposed by enabling the
dotfile debug mode like this:
+
+ // Requires the output folder to be set.
+ DotfileUtills.enableDebugDotfiles(conf);
+This will produce (and save) 4 additional diagrams that visualize the internal
stages of the pipeline execution plan. Such diagrams are the PCollection
pineage, the pipeline base and split graphs and the run-time node (RTNode)
representation.
+(Note: The debug mode requires the output folder to be set. )
+
\ No newline at end of file
