[
https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15667393#comment-15667393
]
Aihua Xu commented on HIVE-14162:
---------------------------------
[~xuefuz] My first patch doesn't work. I'm thinking of creating a new
JobContextImpl (so new SparkContext) for each new job in RemoteDriver rather
than sharing for this jira. After the job is done, the SparkContext should be
dropped. Of course, this will be controlled by a configuration. This
implementation will have performance impact since they are not sharing resource
now, but seems it's good for releasing the resource. Also maybe help to
diagnostics since the executions and the logs are separate. Do you see any
issues with that approach?
> Allow disabling of long running job on Hive On Spark On YARN
> ------------------------------------------------------------
>
> Key: HIVE-14162
> URL: https://issues.apache.org/jira/browse/HIVE-14162
> Project: Hive
> Issue Type: New Feature
> Components: Spark
> Reporter: Thomas Scott
> Assignee: Aihua Xu
> Priority: Minor
> Attachments: HIVE-14162.1.patch
>
>
> Hive On Spark launches a long running process on the first query to handle
> all queries for that user session. In some use cases this is not desired, for
> instance when using Hue with large intervals between query executions.
> Could we have a property that would cause long running spark jobs to be
> terminated after each query execution and started again for the next one?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
