----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63382/#review195265 -----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HiveSplitGenerator.java Lines 224 (patched) <https://reviews.apache.org/r/63382/#comment274445> make sense. I will probably assume the fileSplit.getPath() is a file path and not directory path. I guess it does not matter from trigger's perspective. In most cases, fileSplit.getPath() will be a file except for ACID deltas. Will update the counter to assume getPath() returns file. ql/src/java/org/apache/hadoop/hive/ql/exec/tez/monitoring/TezJobMonitor.java Lines 306 (patched) <https://reviews.apache.org/r/63382/#comment274446> In the context of triggers, we are more interested in *any* vertex violating the constraint. INPUT_DIRECTORIES_vertex_a = 50 INPUT_DIRECTORIES_vertex_b = 100 This will only send INPUT_DIRECTORIES = 100 to trigger validator. We should probably special case this to something like TOTAL_INPUT_DIRECTORIES (aggregated of all vertex counters) MAX_INPUT_DIRECTORIES (max of all vertex counters) Any thoughts? - Prasanth_J On Jan. 11, 2018, 10:17 p.m., Prasanth_J wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63382/ > ----------------------------------------------------------- > > (Updated Jan. 11, 2018, 10:17 p.m.) > > > Review request for hive and Sergey Shelukhin. > > > Bugs: HIVE-17833 > https://issues.apache.org/jira/browse/HIVE-17833 > > > Repository: hive-git > > > Description > ------- > > HIVE-17833: Publish split generation counters > > > Diffs > ----- > > > itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestTriggersTezSessionPoolManager.java > 3b6eb71 > itests/src/test/resources/testconfiguration.properties ac81995 > ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java 92741ee > ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java d68d646 > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkTask.java 6915cf1 > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HiveInputCounters.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HiveSplitGenerator.java > 5dd5e80 > > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/monitoring/TezJobMonitor.java > 5ade1f3 > ql/src/java/org/apache/hadoop/hive/ql/hooks/PostExecTezSummaryPrinter.java > 45bd6e0 > ql/src/test/queries/clientpositive/tez_input_counters.q PRE-CREATION > ql/src/test/results/clientpositive/llap/tez_input_counters.q.out > PRE-CREATION > > > Diff: https://reviews.apache.org/r/63382/diff/2/ > > > Testing > ------- > > > Thanks, > > Prasanth_J > >
