deniskuzZ commented on code in PR #4043:
URL: https://github.com/apache/hive/pull/4043#discussion_r1388381233
##########
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ParallelEdgeFixer.java:
##########
@@ -324,26 +330,25 @@ private static String extractColumnName(ExprNodeDesc
expr) throws SemanticExcept
public static Optional<Set<String>> colMappingInverseKeys(ReduceSinkOperator
rs) {
Map<String, String> ret = new HashMap<String, String>();
Map<String, ExprNodeDesc> exprMap = rs.getColumnExprMap();
- Set<String> neededColumns = new HashSet<String>();
+
+ if
(!rs.getSchema().getColumnNames().stream().allMatch(exprMap::containsKey)) {
+ // Cannot invert RS because exprMap does not contain all of the RS's
input columns.
+ return Optional.empty();
+ }
+
try {
for (Entry<String, ExprNodeDesc> e : exprMap.entrySet()) {
String columnName = extractColumnName(e.getValue());
if (rs.getSchema().getColumnInfo(e.getKey()) == null) {
- // ignore incorrectly mapped columns (if there's any) - but require
its input to be present
- neededColumns.add(columnName);
- } else {
- ret.put(columnName, e.getKey());
+ // Cannot invert RS because the column needed by expression is not
mapped correctly.
+ return Optional.empty();
Review Comment:
it looks like `exprMap` might have duplicate columns {columnName} with
different mappings {e.getKey()}, so I think we need to restore to prev version
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
