> On April 18, 2017, 11:13 p.m., Jason Dere wrote: > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java > > Lines 171 (patched) > > <https://reviews.apache.org/r/58390/diff/2/?file=1692285#file1692285line176> > > > > It should be valid to remove the synthetic predicate here. The NOT NULL > > predicate is probably added to the filter due to it being an inner join. > > Remove this comment/question.
Will do that. > On April 18, 2017, 11:13 p.m., Jason Dere wrote: > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java > > Line 224 (original), 224 (patched) > > <https://reviews.apache.org/r/58390/diff/2/?file=1692285#file1692285line229> > > > > The semijoin hints do not work if the semijoin is generated from the > > result of a join. > > > > One thing that could be useful here is > > ExprNodeDescUtils.findColumnOrigin(), which should be able to trace the > > origin of a column to it's original table operator (assuming it's been > > implemented properly). This might be useful if we want the hints to work > > even with join results. Sounds promising. I will investigate. > On April 18, 2017, 11:13 p.m., Jason Dere wrote: > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java > > Lines 225 (patched) > > <https://reviews.apache.org/r/58390/diff/2/?file=1692285#file1692285line230> > > > > The current semijoin hint mechanism allows either a single column from > > a table, or potentially any column from a table. Would we be able to create > > a semijoin hint for more than one specific column in a table? As of now, no. Incase of n-way join or join involving composite key, user can provide multiple hints. > On April 18, 2017, 11:13 p.m., Jason Dere wrote: > > ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java > > Lines 8124 (patched) > > <https://reviews.apache.org/r/58390/diff/2/?file=1692292#file1692292line8124> > > > > The hint information is being added to the JoinDesc of the Join > > operator .. do we need to propagate this information if the join is > > replaced by a MapJoin in ConvertJoinMapJoin? It should be. I will investigate. - Deepak ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58390/#review172266 ----------------------------------------------------------- On April 13, 2017, 9:18 p.m., Deepak Jaiswal wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58390/ > ----------------------------------------------------------- > > (Updated April 13, 2017, 9:18 p.m.) > > > Review request for hive, Jason Dere and Sergey Shelukhin. > > > Bugs: HIVE-16423 > https://issues.apache.org/jira/browse/HIVE-16423 > > > Repository: hive-git > > > Description > ------- > > De-duplicate semijoin branches and add hint to enforce semi join optimization > > > Diffs > ----- > > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 726522bd94 > itests/src/test/resources/testconfiguration.properties fb85b9ece2 > ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java > db6b05b7ac > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java > 838cc69cb6 > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/HiveOpConverter.java > 73a9b0f34d > ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 79662ec5f1 > ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java d58f4474b6 > ql/src/java/org/apache/hadoop/hive/ql/parse/HintParser.g 8e70a46235 > ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 3f9f76c57f > ql/src/java/org/apache/hadoop/hive/ql/parse/QBJoinTree.java ec76fb79c5 > ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java > e27e953d76 > ql/src/java/org/apache/hadoop/hive/ql/parse/SemiJoinBranchInfo.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/parse/SemiJoinHint.java PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 5f9ccc8ceb > ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java abf531b259 > ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDynamicListDesc.java > 18e4fbdc0a > ql/src/java/org/apache/hadoop/hive/ql/plan/JoinDesc.java bcf3691342 > ql/src/java/org/apache/hadoop/hive/ql/ppd/SyntheticJoinPredicate.java > 71c731081f > > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBloomFilter.java > 2b84beb100 > ql/src/test/queries/clientpositive/semijoin_hint.q PRE-CREATION > ql/src/test/results/clientpositive/llap/semijoin_hint.q.out PRE-CREATION > > > Diff: https://reviews.apache.org/r/58390/diff/2/ > > > Testing > ------- > > > Thanks, > > Deepak Jaiswal > >
