[
https://issues.apache.org/jira/browse/HIVE-20790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662428#comment-16662428
]
Sahil Takiar commented on HIVE-20790:
-------------------------------------
As a follow up to HIVE-20737, we also want to support: "Make getting an opened
Spark Session + submitting a Spark job an atomic operation". This avoids the
edge case where a user tries to submit a query while the session is being
closed. There is a race condition where the code might see that isOpen is true,
but when it tries to submit the query the session has already been closed by
another thread. This will cause the submission to fail with an
{{IllegalStateException}}. Making the check of isOpen and the submission atomic
avoids this scenario. Essentially, we want to support valid semantics when a
user submits a query while the session is being closed. So if a user submits a
query while a job is being closed, it waits for the close to complete and then
re-opens the session and submits the job.
> SparkSession should be able to close a session while it is being opened
> -----------------------------------------------------------------------
>
> Key: HIVE-20790
> URL: https://issues.apache.org/jira/browse/HIVE-20790
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
> Reporter: Sahil Takiar
> Assignee: Antal Sinkovits
> Priority: Major
>
> In HIVE-14162 we adding locks to {{SparkSessionImpl}} to support scenarios
> where we want to close the session due to a timeout. However, the locks
> remove the ability to close a session while it is being opened. This is
> important to allow cancelling of a session while it is being setup. This can
> be useful on busy clusters where there may not be enough YARN containers to
> setup the Spark Remote Driver.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
