juripetersen opened a new pull request, #421:
URL: https://github.com/apache/incubator-wayang/pull/421
References #417
Users can define WayangPlans as usual, but instead of executing them, they
can now use an explain command:
```java
// Debug into console
WayangContext.explain(wayangPlan, udfJars);
// Print to json files for later usage:
WayangContext.explain(wayangPlan, true, udfJars);
```
Example debug to console for WordCount:
```shell
== Wayang Plan ==
-+ [Alternative[[LocalCallbackSink[Collect result]]],
Alternative[[JavaLocalCallbackSink[Collect result]]],
Alternative[[SparkLocalCallbackSink[Collect result]]]]
-+ [Alternative[[ReduceBy[Add counters]]], Alternative[[JavaReduceBy[Add
counters]]], Alternative[[SparkReduceBy[Add counters]]]]
-+ [Alternative[[Map[To lower case, add counter]]],
Alternative[[JavaMap[To lower case, add counter]]], Alternative[[SparkMap[To
lower case, add counter]]]]
-+ [Alternative[[Filter[Filter empty words]]],
Alternative[[JavaFilter[Filter empty words]]], Alternative[[SparkFilter[Filter
empty words]]]]
-+ [Alternative[[FlatMap[Split words]]],
Alternative[[JavaFlatMap[Split words]]], Alternative[[SparkFlatMap[Split
words]]]]
-+ [Alternative[[TextFileSource[Load file]]],
Alternative[[JavaTextFileSource[Load file]]],
Alternative[[SparkTextFileSource[Load file]]]]
== Execution Plan ==
-+ JavaLocalCallbackSink[Collect result]
-+ JavaReduceBy[Add counters]
-+ JavaMap[To lower case, add counter]
-+ JavaFilter[Filter empty words]
-+ JavaFlatMap[Split words]
-+ JavaTextFileSource[Load file]
```
The class `ExplainUtils` was added that allows to retrieve
`ExplainTreeNodes` that can be used for further computations (maybe other file
formats etc.). This class allows to specify upstream or downstream traversal of
plans so that users can build trees that fit their needs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
