----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63382/#review195264 -----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HiveSplitGenerator.java Lines 224 (patched) <https://reviews.apache.org/r/63382/#comment274443> hmm... does this iterate thru the entire input of the query? That can be very expensive on cloud FS or if there are millions of partitions. I don't think we should do that ql/src/java/org/apache/hadoop/hive/ql/exec/tez/monitoring/TezJobMonitor.java Lines 306 (patched) <https://reviews.apache.org/r/63382/#comment274444> it seems like counter name without vertex name would imply total, not max. I.e. if I say splits > 50, or input-size > 1Gb I mean total splits > 50/size > 1Gb, not for any one vertex. - Sergey Shelukhin On Jan. 11, 2018, 10:17 p.m., Prasanth_J wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63382/ > ----------------------------------------------------------- > > (Updated Jan. 11, 2018, 10:17 p.m.) > > > Review request for hive and Sergey Shelukhin. > > > Bugs: HIVE-17833 > https://issues.apache.org/jira/browse/HIVE-17833 > > > Repository: hive-git > > > Description > ------- > > HIVE-17833: Publish split generation counters > > > Diffs > ----- > > > itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestTriggersTezSessionPoolManager.java > 3b6eb71 > itests/src/test/resources/testconfiguration.properties ac81995 > ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java 92741ee > ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java d68d646 > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkTask.java 6915cf1 > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HiveInputCounters.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HiveSplitGenerator.java > 5dd5e80 > > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/monitoring/TezJobMonitor.java > 5ade1f3 > ql/src/java/org/apache/hadoop/hive/ql/hooks/PostExecTezSummaryPrinter.java > 45bd6e0 > ql/src/test/queries/clientpositive/tez_input_counters.q PRE-CREATION > ql/src/test/results/clientpositive/llap/tez_input_counters.q.out > PRE-CREATION > > > Diff: https://reviews.apache.org/r/63382/diff/2/ > > > Testing > ------- > > > Thanks, > > Prasanth_J > >
