[
https://issues.apache.org/jira/browse/HIVE-17291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16126224#comment-16126224
]
Peter Vary commented on HIVE-17291:
-----------------------------------
To clarify: I see the flakiness appear only when running the HoS tests against
*real clusters*. I am still not sure why I do not see it with [~lirui]'s magic.
When running the tests agains an appropriately configured HoS cluster I am
using the TestBeeLineDriver to run the query tests, and I see the flakiness
there. The BeeLineDriver could not use the QTestUtil magic, since it runs the
query against a real HS2 instance with the out of the box Sessions. So the
current state is not that bad!
Quick question: Do we have a way to determine when info about the available
executors is reliable? The current implementation assumes that if we have a
value which is bigger than 0, then it is reliable. But it still might be
invalid if we requested multiple executors, but not yet received all.
Thanks,
Peter
> Set the number of executors based on config if client does not provide
> information
> ----------------------------------------------------------------------------------
>
> Key: HIVE-17291
> URL: https://issues.apache.org/jira/browse/HIVE-17291
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
> Affects Versions: 3.0.0
> Reporter: Peter Vary
> Assignee: Peter Vary
> Attachments: HIVE-17291.1.patch
>
>
> When calculating the memory and cores and the client does not provide
> information we should try to use the one provided by default. This can happen
> on startup, when {{spark.dynamicAllocation.enabled}} is not enabled
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
