> On Oct. 22, 2013, 12:19 a.m., Daniel Dai wrote: > > src/org/apache/pig/backend/hadoop/executionengine/tez/TezLauncher.java, > > line 50 > > <https://reviews.apache.org/r/14776/diff/2/?file=368517#file368517line50> > > > > We will soon need to launch multiple jobs from Pig, and collect and > > aggregate multiple TezStats.
Thank you Daniel for the review! I can look into handling multiple DAGs. It will require changes in TezJobControlCompiler as well. - Cheolsoo ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14776/#review27266 ----------------------------------------------------------- On Oct. 20, 2013, 11:06 p.m., Cheolsoo Park wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/14776/ > ----------------------------------------------------------- > > (Updated Oct. 20, 2013, 11:06 p.m.) > > > Review request for pig, Daniel Dai, Mark Wagner, and Rohini Palaniswamy. > > > Bugs: PIG-3514 > https://issues.apache.org/jira/browse/PIG-3514 > > > Repository: pig-git > > > Description > ------- > > This is an initial implementation of TezStats. For now, it collects the > number of succeeded/failed Tez vertices and prints the "success/failed" > message at the end. > > In summary, I implemented the following classes: > * TezStats extends PigStats > * TezVertexStats extends JobStats > > Note that TezVertexStats captures a Tez vertex not a Tez job. > > In addition, I moved several fields and methods that can be commonly used by > both SimplePigStats and TezStats to PigStats. > > > Diffs > ----- > > src/org/apache/pig/backend/hadoop/executionengine/tez/TezExecType.java > c726923 > > src/org/apache/pig/backend/hadoop/executionengine/tez/TezExecutionEngine.java > 6e748a8 > src/org/apache/pig/backend/hadoop/executionengine/tez/TezJob.java d88eb6b > src/org/apache/pig/backend/hadoop/executionengine/tez/TezLauncher.java > d7577c1 > src/org/apache/pig/backend/hadoop/executionengine/tez/TezScriptState.java > eb4eefb > src/org/apache/pig/backend/hadoop/executionengine/tez/TezStats.java ef8733a > src/org/apache/pig/tools/pigstats/PigStats.java 6d2e58e > src/org/apache/pig/tools/pigstats/mapreduce/SimplePigStats.java bbfd5a9 > src/org/apache/pig/tools/pigstats/tez/TezScriptState.java e69de29 > src/org/apache/pig/tools/pigstats/tez/TezStats.java e69de29 > src/org/apache/pig/tools/pigstats/tez/TezTaskStats.java e69de29 > test/org/apache/pig/tez/TestTezLauncher.java 0a3bc73 > > Diff: https://reviews.apache.org/r/14776/diff/ > > > Testing > ------- > > Ran a MRR job on a single node Tez cluster and confirmed that the job status > is printed correctly. Here are examples: > > ----- > 2013-10-20 00:29:13,218 [main] INFO > org.apache.pig.tools.pigstats.tez.TezStats - Script Statistics: > > HadoopVersion PigVersion UserId StartedAt FinishedAt Features > 2.2.0 0.13.0-SNAPSHOT cheolsoop 2013-10-20 00:28:44 2013-10-20 > 00:29:13 GROUP_BY,FILTER > > Success! > ----- > 2013-10-20 00:30:10,970 [main] INFO > org.apache.pig.tools.pigstats.tez.TezStats - Script Statistics: > > HadoopVersion PigVersion UserId StartedAt FinishedAt Features > 2.2.0 0.13.0-SNAPSHOT cheolsoop 2013-10-20 00:29:30 2013-10-20 > 00:30:10 GROUP_BY,FILTER > > Failed! > ----- > > More unit tests will be added after Tez mini cluster is added. > > > Thanks, > > Cheolsoo Park > >
