-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14776/
-----------------------------------------------------------
(Updated Oct. 20, 2013, 11:06 p.m.)
Review request for pig, Daniel Dai, Mark Wagner, and Rohini Palaniswamy.
Changes
-------
Minor updates:
* Moved TezScriptState to pig.stats.tez wherein TezStats and TezTaskStats are.
* Renamed TezVertexStats to TezTaskStats.
* Cleaned up the TezStats format.
* Added Tez version to TezStats.
Here is the new TezStats output:
2013-10-20 15:52:39,219 [main] INFO org.apache.pig.tools.pigstats.tez.TezStats
- Script Statistics:
HadoopVersion: 2.2.0
TezVersion: 0.2.0-SNAPSHOT
PigVersion: 0.13.0-SNAPSHOT
UserId: cheolsoop
StartedAt: 2013-10-20 15:52:10
FinishedAt: 2013-10-20 15:52:39
Features: GROUP_BY,FILTER
Success!
Bugs: PIG-3514
https://issues.apache.org/jira/browse/PIG-3514
Repository: pig-git
Description
-------
This is an initial implementation of TezStats. For now, it collects the number
of succeeded/failed Tez vertices and prints the "success/failed" message at the
end.
In summary, I implemented the following classes:
* TezStats extends PigStats
* TezVertexStats extends JobStats
Note that TezVertexStats captures a Tez vertex not a Tez job.
In addition, I moved several fields and methods that can be commonly used by
both SimplePigStats and TezStats to PigStats.
Diffs (updated)
-----
src/org/apache/pig/backend/hadoop/executionengine/tez/TezExecType.java
c726923
src/org/apache/pig/backend/hadoop/executionengine/tez/TezExecutionEngine.java
6e748a8
src/org/apache/pig/backend/hadoop/executionengine/tez/TezJob.java d88eb6b
src/org/apache/pig/backend/hadoop/executionengine/tez/TezLauncher.java
d7577c1
src/org/apache/pig/backend/hadoop/executionengine/tez/TezScriptState.java
eb4eefb
src/org/apache/pig/backend/hadoop/executionengine/tez/TezStats.java ef8733a
src/org/apache/pig/tools/pigstats/PigStats.java 6d2e58e
src/org/apache/pig/tools/pigstats/mapreduce/SimplePigStats.java bbfd5a9
src/org/apache/pig/tools/pigstats/tez/TezScriptState.java e69de29
src/org/apache/pig/tools/pigstats/tez/TezStats.java e69de29
src/org/apache/pig/tools/pigstats/tez/TezTaskStats.java e69de29
test/org/apache/pig/tez/TestTezLauncher.java 0a3bc73
Diff: https://reviews.apache.org/r/14776/diff/
Testing
-------
Ran a MRR job on a single node Tez cluster and confirmed that the job status is
printed correctly. Here are examples:
-----
2013-10-20 00:29:13,218 [main] INFO org.apache.pig.tools.pigstats.tez.TezStats
- Script Statistics:
HadoopVersion PigVersion UserId StartedAt FinishedAt Features
2.2.0 0.13.0-SNAPSHOT cheolsoop 2013-10-20 00:28:44 2013-10-20
00:29:13 GROUP_BY,FILTER
Success!
-----
2013-10-20 00:30:10,970 [main] INFO org.apache.pig.tools.pigstats.tez.TezStats
- Script Statistics:
HadoopVersion PigVersion UserId StartedAt FinishedAt Features
2.2.0 0.13.0-SNAPSHOT cheolsoop 2013-10-20 00:29:30 2013-10-20
00:30:10 GROUP_BY,FILTER
Failed!
-----
More unit tests will be added after Tez mini cluster is added.
Thanks,
Cheolsoo Park
