Adam Szita created HIVE-21176:
---------------------------------
Summary: SetSparkReducerParallelism should check
spark.executor.instances before opening SparkSession during compilation
Key: HIVE-21176
URL: https://issues.apache.org/jira/browse/HIVE-21176
Project: Hive
Issue Type: Bug
Reporter: Adam Szita
Assignee: Adam Szita
{{SetSparkReducerParallelism}} creates a spark session in the compilation stage
while holding the compile lock. This is a very expensive operation and can
cause a complete slowdown of all the hive queries. The problem only occurs when
dynamicAllocation is disabled, but we should find a way to improve this:
e.g. if spark.executor.instances is set we already know how many executors will
be launched
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
