----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18832/#review36625 -----------------------------------------------------------
Ship it! Ship It! - Rohini Palaniswamy On March 6, 2014, 8 a.m., Cheolsoo Park wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/18832/ > ----------------------------------------------------------- > > (Updated March 6, 2014, 8 a.m.) > > > Review request for pig, Mark Wagner and Rohini Palaniswamy. > > > Bugs: PIG-3603 > https://issues.apache.org/jira/browse/PIG-3603 > > > Repository: pig-git > > > Description > ------- > > This patch adds the following counters to TezStats: > 1) # of input/output records > 2) hdfs bytes read/written > 3) file bytes read/written > > The job stats looks like this: > > JobId: job_pigexec_0 > > TotalLaunchedTasks: 3 > > FileBytesRead: 2434726 > > FileBytesWritten: 4869516 > > HdfsBytesRead: 2219954 > > HdfsBytesWritten: 2433980 > > > Input(s): Successfully read 10000 records (1109977 bytes) from: > "/user/pig/tests/data/singlefile/studentcomplextab10k" > : Successfully read 10000 records (1109977 bytes) from: > "/user/pig/tests/data/singlefile/studentcomplextab10k" > Output(s): Successfully stored 10393 records (2433980 bytes) in: > "hdfs://localhost:57063/tmp/temp90703803/tmp-1606775243" > > This patch also includes the following changes in PigStats/JobStats classes: > 1) Move getHdfsBytesRead() and getHdfsBytesWritten() from MRPigStatsUtil to > PigStatsUtil since these are not MR specific. > 2) Move [MAP|REDUCE]_[OUT|IN]PUT_RECORDS from MRPigStatsUtil to PigStatsUtil > since Tez MRInput and MROutput also use them. > 3) Fix a typo in JobStats#getAvgREduceTime(): REduce -> Reduce. > 4) Fix white spaces. > > Note that none of these changes breaks backward compatibility. > > > Diffs > ----- > > src/org/apache/pig/PigServer.java 2004edb > src/org/apache/pig/backend/hadoop/executionengine/tez/TezJob.java 5d12091 > src/org/apache/pig/tools/pigstats/InputStats.java 38c8372 > src/org/apache/pig/tools/pigstats/JobStats.java 4484348 > src/org/apache/pig/tools/pigstats/OutputStats.java 6a3e3eb > src/org/apache/pig/tools/pigstats/PigStats.java 3032728 > src/org/apache/pig/tools/pigstats/PigStatsUtil.java e690b8d > src/org/apache/pig/tools/pigstats/ScriptState.java d58310d > src/org/apache/pig/tools/pigstats/mapreduce/MRJobStats.java 115ae1d > src/org/apache/pig/tools/pigstats/mapreduce/MRPigStatsUtil.java ed791fd > src/org/apache/pig/tools/pigstats/tez/TezStats.java 64d70e7 > src/org/apache/pig/tools/pigstats/tez/TezTaskStats.java c3f1c3e > test/org/apache/pig/test/TestCombiner.java ae2135e > test/org/apache/pig/test/TestPigServer.java 8613c3b > > Diff: https://reviews.apache.org/r/18832/diff/ > > > Testing > ------- > > ant test-tez passes except TestTezCompiler (known). > tez e2e tests pass. > > > Thanks, > > Cheolsoo Park > >
