[
https://issues.apache.org/jira/browse/HIVE-23831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated HIVE-23831:
--------------------------------
Summary: Make auto reducer parallelism min threshold configurable. (was:
Introduce a threshold to turn on or off auto-parallelism of Tez)
> Make auto reducer parallelism min threshold configurable.
> ---------------------------------------------------------
>
> Key: HIVE-23831
> URL: https://issues.apache.org/jira/browse/HIVE-23831
> Project: Hive
> Issue Type: Improvement
> Components: Hive
> Reporter: okumin
> Assignee: okumin
> Priority: Major
> Labels: pull-request-available
>
> Hive on Tez disables auto-reduce and uses the maximum number of partitions
> when the minimum number of reducers, `estimated number *
> min.partition.factor`, is less than or equal to 1.
> [https://github.com/apache/hive/blob/f98e136bdd5642e3de10d2fd1a4c14d1d6762113/ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java#L121-L131]
> https://issues.apache.org/jira/browse/HIVE-14200
>
> This can make Tez lose chance to optimize # of reducers especially in an
> environment with small `hive.tez.min.partition.factor` or big
> `hive.tez.max.partition.factor`.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
