deniskuzZ commented on code in PR #4043:
URL: https://github.com/apache/hive/pull/4043#discussion_r1388649716
##########
ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java:
##########
@@ -678,38 +678,34 @@ private boolean
generateSemiJoinOperatorPlan(DynamicListContext ctx, ParseContex
ArrayList<ColumnInfo> groupbyColInfos = new ArrayList<ColumnInfo>();
groupbyColInfos.add(new ColumnInfo(gbOutputNames.get(0),
key.getTypeInfo(), "", false));
groupbyColInfos.add(new ColumnInfo(gbOutputNames.get(1),
key.getTypeInfo(), "", false));
- groupbyColInfos.add(new ColumnInfo(gbOutputNames.get(2),
key.getTypeInfo(), "", false));
+ groupbyColInfos.add(new ColumnInfo(gbOutputNames.get(2),
TypeInfoFactory.binaryTypeInfo, "", false));
GroupByOperator groupByOp =
(GroupByOperator)OperatorFactory.getAndMakeChild(
groupBy, new RowSchema(groupbyColInfos), selectOp);
groupByOp.setColumnExprMap(new HashMap<String, ExprNodeDesc>());
// Get the column names of the aggregations for reduce sink
- int colPos = 0;
ArrayList<ExprNodeDesc> rsValueCols = new ArrayList<ExprNodeDesc>();
Map<String, ExprNodeDesc> columnExprMap = new HashMap<String,
ExprNodeDesc>();
- for (int i = 0; i < aggs.size() - 1; i++) {
- ExprNodeColumnDesc colExpr = new ExprNodeColumnDesc(key.getTypeInfo(),
- gbOutputNames.get(colPos), "", false);
+ ArrayList<ColumnInfo> rsColInfos = new ArrayList<>();
+ for (int colPos = 0; colPos < aggs.size(); colPos++) {
+ TypeInfo typInfo = groupbyColInfos.get(colPos).getType();
+ ExprNodeColumnDesc colExpr = new ExprNodeColumnDesc(typInfo,
gbOutputNames.get(colPos), "", false);
rsValueCols.add(colExpr);
- columnExprMap.put(gbOutputNames.get(colPos), colExpr);
- colPos++;
- }
+ columnExprMap.put(Utilities.ReduceField.VALUE + "." +
gbOutputNames.get(colPos), colExpr);
- // Bloom Filter uses binary
- ExprNodeColumnDesc colExpr = new
ExprNodeColumnDesc(TypeInfoFactory.binaryTypeInfo,
- gbOutputNames.get(colPos), "", false);
- rsValueCols.add(colExpr);
- columnExprMap.put(gbOutputNames.get(colPos), colExpr);
- colPos++;
+ ColumnInfo colInfo =
Review Comment:
I've removed the ReduceField.VALUE from generateSemiJoinOperatorPlan, but
kept in createFinalRsForSemiJoinOp --> sharedwork_semi_2.q passed and produced.
same result
````
**generateSemiJoinOperatorPlan**
for (int i = 0; i < aggs.size(); i++) {
ExprNodeColumnDesc colExpr = new
ExprNodeColumnDesc(groupbyColInfos.get(i).getType(),
gbOutputNames.get(i), "", false);
rsValueCols.add(colExpr);
columnExprMap.put(gbOutputNames.get(i), colExpr);
}
**createFinalRsForSemiJoinOp**
Map<String, ExprNodeDesc> columnExprMap = new HashMap<>();
ArrayList<ColumnInfo> gbySchema = new ArrayList<>();
for (ColumnInfo rsColInfo : gb.getSchema().getSignature()) {
gbOutputNames.add(rsColInfo.getInternalName());
ExprNodeColumnDesc rsValExpr = new
ExprNodeColumnDesc(rsColInfo.getType(), rsColInfo.getInternalName(), "", false);
rsValueCols.add(rsValExpr);
String field = Utilities.ReduceField.VALUE.toString() + "." +
outputColName;
columnExprMap.put(field, rsValExpr);
rsColInfos.add(new ColumnInfo(field, rsColInfo.getType(), "", false));
}
````
also that didn't require changes in ParallelEdgeFixer.colMappingInverseKeys,
exprMap wasn't empty
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
