jcamachor commented on a change in pull request #1152:
URL: https://github.com/apache/hive/pull/1152#discussion_r443271395
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java
##########
@@ -1566,13 +1569,38 @@ private void
removeSemijoinsParallelToMapJoin(OptimizeTezProcContext procCtx)
List<ExprNodeDesc> keyDesc =
selectedMJOp.getConf().getKeys().get(posBigTable);
ExprNodeColumnDesc keyCol = (ExprNodeColumnDesc) keyDesc.get(0);
-
- tsProbeDecodeCtx = new TableScanOperator.ProbeDecodeContext(mjCacheKey,
mjSmallTablePos,
- keyCol.getColumn(), selectedMJOpRatio);
+ String realTSColName = getOriginalTSColName(selectedMJOp,
keyCol.getColumn());
+ if (realTSColName != null) {
+ tsProbeDecodeCtx = new
TableScanOperator.ProbeDecodeContext(mjCacheKey, mjSmallTablePos,
+ realTSColName, selectedMJOpRatio);
+ } else {
+ LOG.warn("ProbeDecode could not find TSColName for ColKey {} with MJ
Schema {} ", keyCol, selectedMJOp.getSchema());
+ }
}
return tsProbeDecodeCtx;
}
+ private static String getOriginalTSColName(MapJoinOperator mjOp, String
internalCoName) {
Review comment:
Can you check if this can already be done by any of the utility methods
in `OperatorUtils`? If it is not, can we move it to that class with rest of
utility methods in case it is useful in the future?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java
##########
@@ -1566,13 +1569,38 @@ private void
removeSemijoinsParallelToMapJoin(OptimizeTezProcContext procCtx)
List<ExprNodeDesc> keyDesc =
selectedMJOp.getConf().getKeys().get(posBigTable);
ExprNodeColumnDesc keyCol = (ExprNodeColumnDesc) keyDesc.get(0);
-
- tsProbeDecodeCtx = new TableScanOperator.ProbeDecodeContext(mjCacheKey,
mjSmallTablePos,
- keyCol.getColumn(), selectedMJOpRatio);
+ String realTSColName = getOriginalTSColName(selectedMJOp,
keyCol.getColumn());
+ if (realTSColName != null) {
+ tsProbeDecodeCtx = new
TableScanOperator.ProbeDecodeContext(mjCacheKey, mjSmallTablePos,
+ realTSColName, selectedMJOpRatio);
+ } else {
+ LOG.warn("ProbeDecode could not find TSColName for ColKey {} with MJ
Schema {} ", keyCol, selectedMJOp.getSchema());
Review comment:
You could throw an error only if you are running in test mode:
```
if (conf.getBoolVar(ConfVars.HIVE_IN_TEST)) {
...
```
While I am not a huge fan of interleaving a check like this within the
production code, it may help you identify any issues, gaps, or regressions in
the future.
If you do not want to do that, an alternative is uploading a patch throwing
an error instead of printing the warning to get a complete test run, then
create follow-up issues if there are any. It will not help you with regressions
in the future, but you will be able to identify any existing issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
