Gopal V created HIVE-20973: ------------------------------ Summary: Optimizer: Reduce de-dup changes the hash-function of a reducer edge Key: HIVE-20973 URL: https://issues.apache.org/jira/browse/HIVE-20973 Project: Hive Issue Type: Bug Reporter: Gopal V
{code} private static void propagateMaxNumReducers(ReduceSinkJoinDeDuplicateProcCtx dedupCtx, ReduceSinkOperator rsOp, int maxNumReducers) throws SemanticException { if (rsOp == null) { // Bail out return; } if (rsOp.getChildOperators().get(0) instanceof MapJoinOperator || rsOp.getChildOperators().get(0) instanceof CommonMergeJoinOperator) { for (Operator<?> p : rsOp.getChildOperators().get(0).getParentOperators()) { ReduceSinkOperator pRSOp = (ReduceSinkOperator) p; pRSOp.getConf().setReducerTraits(EnumSet.of(ReducerTraits.FIXED)); pRSOp.getConf().setNumReducers(maxNumReducers); LOG.debug("Set {} to FIXED parallelism: {}", pRSOp, maxNumReducers); if (pRSOp.getConf().isForwarding()) { ReduceSinkOperator newRSOp = CorrelationUtilities.findFirstPossibleParent( pRSOp, ReduceSinkOperator.class, dedupCtx.trustScript()); propagateMaxNumReducers(dedupCtx, newRSOp, maxNumReducers); } } {code} FIXED used to mean AUTOPARALLEL=false, but now FIXED means a different hash function from UNIFORM. -- This message was sent by Atlassian JIRA (v7.6.3#76005)