shameersss1 commented on a change in pull request #1400:
URL: https://github.com/apache/hive/pull/1400#discussion_r479894128
##########
File path:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/correlation/ReduceSinkDeDuplicationUtils.java
##########
@@ -181,6 +183,23 @@ public static boolean merge(HiveConf hiveConf,
ReduceSinkOperator cRS, ReduceSin
TableDesc keyTable = PlanUtils.getReduceKeyTableDesc(new
ArrayList<FieldSchema>(), pRS
.getConf().getOrder(), pRS.getConf().getNullOrder());
pRS.getConf().setKeySerializeInfo(keyTable);
+ } else if (cRS.getConf().getKeyCols() != null &&
cRS.getConf().getKeyCols().size() > 0) {
+ ArrayList<String> keyColNames = Lists.newArrayList();
+ for (ExprNodeDesc keyCol : pRS.getConf().getKeyCols()) {
+ String keyColName = keyCol.getExprString();
+ keyColNames.add(keyColName);
+ }
+ List<FieldSchema> fields =
PlanUtils.getFieldSchemasFromColumnList(pRS.getConf().getKeyCols(),
+ keyColNames, 0, "");
+ TableDesc keyTable = PlanUtils.getReduceKeyTableDesc(fields,
pRS.getConf().getOrder(),
+ pRS.getConf().getNullOrder());
+ ArrayList<String> outputKeyCols = Lists.newArrayList();
+ for (int i = 0; i < fields.size(); i++) {
+ outputKeyCols.add(fields.get(i).getName());
+ }
+ pRS.getConf().setOutputKeyColumnNames(outputKeyCols);
+ pRS.getConf().setKeySerializeInfo(keyTable);
+
pRS.getConf().setNumDistributionKeys(cRS.getConf().getNumDistributionKeys());
}
Review comment:
Such case would arise only when both pRS keyCol is not empty and cRS
keyCol is empty, In such cases wouldn't it be correct to return to true and go
with the pRS values. I mean by the time the program pointer reaches here there
would have been some merging of cRS to pRS would have happened upstream.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
