----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/65422/#review197649 -----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/AbstractReExecDriver.java Lines 131 (patched) <https://reviews.apache.org/r/65422/#comment277871> This is hackish.. as pointed above it needs to happen via explain modifier. ql/src/java/org/apache/hadoop/hive/ql/Context.java Lines 72 (patched) <https://reviews.apache.org/r/65422/#comment277873> Use java.util.Optional instead ql/src/java/org/apache/hadoop/hive/ql/DriverFactory.java Lines 21 (patched) <https://reviews.apache.org/r/65422/#comment277882> Use java's nonnull annotation. ql/src/java/org/apache/hadoop/hive/ql/ReExecOverlayDriver.java Lines 40 (patched) <https://reviews.apache.org/r/65422/#comment277884> Need to check for org.apache.hadoop.hive.ql.exec.mapjoin.MapJoinMemoryExhaustionError and java.lang.OutOfMemoryError only. ql/src/java/org/apache/hadoop/hive/ql/ReOptimizeDriver.java Lines 43 (patched) <https://reviews.apache.org/r/65422/#comment277885> Need to check for org.apache.hadoop.hive.ql.exec.mapjoin.MapJoinMemoryExhaustionError and java.lang.OutOfMemoryError only. ql/src/java/org/apache/hadoop/hive/ql/ReOptimizeDriver.java Lines 54 (patched) <https://reviews.apache.org/r/65422/#comment277886> Why is this needed? ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java Lines 139 (patched) <https://reviews.apache.org/r/65422/#comment277893> Should also compare with parent to check rowcount and other stats are not greater than parent. ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java Lines 310 (patched) <https://reviews.apache.org/r/65422/#comment277894> Should also compare with parent to check rowcount and other stats are not greater than parent and normalize to parent stats in that case. ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java Lines 2314 (patched) <https://reviews.apache.org/r/65422/#comment277895> Should also compare with parent to check rowcount and other stats are not greater than parent and normalize to parent stats in that case. ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/PlanMapper.java Lines 36 (patched) <https://reviews.apache.org/r/65422/#comment277890> A flat map of operators looses hierarichal info in which operators are organized which is tree. So, this match needs to happen via sub-graph matching pattern. See SharedWorkOptimizer::areMergeable() . ql/src/java/org/apache/hadoop/hive/ql/stats/OperatorStatsReaderHook.java Lines 75 (patched) <https://reviews.apache.org/r/65422/#comment277891> LOG.debug ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAssertTrueOOM.java Lines 64 (patched) <https://reviews.apache.org/r/65422/#comment277892> throw MapJoinMemoryExhaustionError - Ashutosh Chauhan On Jan. 30, 2018, 6:13 p.m., Zoltan Haindrich wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/65422/ > ----------------------------------------------------------- > > (Updated Jan. 30, 2018, 6:13 p.m.) > > > Review request for hive and Ashutosh Chauhan. > > > Repository: hive-git > > > Description > ------- > > preview > > > Diffs > ----- > > cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java a78e0c63d7 > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java b7d3e99e1a > hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/HCatCli.java > ad31287879 > hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/HCatDriver.java > 533f0bcd6f > itests/src/test/resources/testconfiguration.properties d86ff58840 > ql/src/java/org/apache/hadoop/hive/ql/AbstractReExecDriver.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/Context.java 820fbf0f58 > ql/src/java/org/apache/hadoop/hive/ql/Driver.java 74595b00f9 > ql/src/java/org/apache/hadoop/hive/ql/DriverFactory.java 49d2bf5f33 > ql/src/java/org/apache/hadoop/hive/ql/IDriver.java 6280be0b08 > ql/src/java/org/apache/hadoop/hive/ql/ReExecOverlayDriver.java PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/ReOptimizeDriver.java PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 76e85636d1 > ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java 199b181290 > ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java > 395a5f450f > > ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkCommonOperator.java > 8dd7cfe58c > > ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkEmptyKeyOperator.java > 134fc0ff0b > > ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkObjectHashOperator.java > 1eb72ce4d9 > > ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkUniformHashOperator.java > 384bd74686 > ql/src/java/org/apache/hadoop/hive/ql/hooks/PrivateHookContext.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java > 190771ea6b > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java > cbadfa4f07 > ql/src/java/org/apache/hadoop/hive/ql/plan/Statistics.java 0057f0c2c6 > ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/EmptyStatsSource.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/GroupTransformer.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/PlanMapper.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/PlanMapperProcess.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/RuntimeStatsSource.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/SimpleRuntimeStatsSource.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/StatsSource.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/refs/OperatorRef.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/processors/CommandProcessorFactory.java > dcf8d31eaf > ql/src/java/org/apache/hadoop/hive/ql/stats/OperatorStats.java PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/stats/OperatorStatsReaderHook.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAssertTrueOOM.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFGetSplits.java > d56002d192 > ql/src/test/org/apache/hadoop/hive/ql/plan/mapping/TestCounterMapping.java > PRE-CREATION > ql/src/test/org/apache/hadoop/hive/ql/plan/mapping/TestReOptimization.java > PRE-CREATION > ql/src/test/queries/clientpositive/retry_failure.q PRE-CREATION > ql/src/test/queries/clientpositive/retry_failure_oom.q PRE-CREATION > ql/src/test/queries/clientpositive/retry_failure_stat_changes.q > PRE-CREATION > ql/src/test/results/clientpositive/llap/retry_failure.q.out PRE-CREATION > ql/src/test/results/clientpositive/llap/retry_failure_oom.q.out > PRE-CREATION > ql/src/test/results/clientpositive/llap/retry_failure_stat_changes.q.out > PRE-CREATION > > > Diff: https://reviews.apache.org/r/65422/diff/1/ > > > Testing > ------- > > > Thanks, > > Zoltan Haindrich > >
