[
https://issues.apache.org/jira/browse/HIVE-21176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Szita reassigned HIVE-21176:
---------------------------------
> SetSparkReducerParallelism should check spark.executor.instances before
> opening SparkSession during compilation
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-21176
> URL: https://issues.apache.org/jira/browse/HIVE-21176
> Project: Hive
> Issue Type: Bug
> Reporter: Adam Szita
> Assignee: Adam Szita
> Priority: Major
>
> {{SetSparkReducerParallelism}} creates a spark session in the compilation
> stage while holding the compile lock. This is a very expensive operation and
> can cause a complete slowdown of all the hive queries. The problem only
> occurs when dynamicAllocation is disabled, but we should find a way to
> improve this:
> e.g. if spark.executor.instances is set we already know how many executors
> will be launched
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
