[
https://issues.apache.org/jira/browse/HIVE-17291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16125842#comment-16125842
]
Peter Vary commented on HIVE-17291:
-----------------------------------
Thanks [~lirui], helpful as always!
What I do not get, why we do not see flakiness on TestMiniSparkOnYarnCliDriver
tests, when the we change a spark configuration value, and the spark session is
invalidated. Until now I thought, that in this case new containers are created
by yarn, and we will have a race condition when checking
{{hiveSparkClient.getExecutorCount()}} again. But obviously this is not the
case, since I do not see any flakiness in the test results in HIVE-17292.
Thanks,
Peter
> Set the number of executors based on config if client does not provide
> information
> ----------------------------------------------------------------------------------
>
> Key: HIVE-17291
> URL: https://issues.apache.org/jira/browse/HIVE-17291
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
> Affects Versions: 3.0.0
> Reporter: Peter Vary
> Assignee: Peter Vary
> Attachments: HIVE-17291.1.patch
>
>
> When calculating the memory and cores and the client does not provide
> information we should try to use the one provided by default. This can happen
> on startup, when {{spark.dynamicAllocation.enabled}} is not enabled
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
