[
https://issues.apache.org/jira/browse/LENS-1150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297831#comment-15297831
]
Rajat Khandelwal commented on LENS-1150:
----------------------------------------
While LENS-518 captures the essential changes for the move to hive2, there are
some minor things that have been missed. Mostly they weren't caught by the test
cases and only occur in specific scenarios.
When executing on Hive Driver, lens opens a session with the hive server. As
discussed in HIVE-13415, Hive sessions are somewhat coupled with the thrift
transport. So when the transport closes, the hive session also gets closed
resulting in the killing of all the operations belonging to the session. Two
workarounds are possible:
* Set that property on Hive Server level. That would leave sessions open and
might cause a session leak -- and hence -- memory leak. We did this to move
forward and find another issue -- mentioned later in this comment.
* Change the code in lens that opens Hive Session to open session with that
property set to false. This will be included in this patch.
There is another issue, that occurs only when Lens and Hive Server are running
on the same machine and with the same user. Lens creates a `CLIService`
instance in `LensServices`. Hive also has an instance of CLIService in its JVM.
The CLIService itself has a SessionManager instance to manage sessions --
including operation logs. On lens restart, that service is stopped and [this
code|https://github.com/apache/hive/blob/master/service/src/java/org/apache/hive/service/cli/session/SessionManager.java#L246]
is called. Now, operation log directory is configured by a property
{noformat}
HIVE_SERVER2_LOGGING_OPERATION_LOG_LOCATION("hive.server2.logging.operation.log.location",
"${system:java.io.tmpdir}" + File.separator + "${system:user.name}" +
File.separator +
"operation_logs",
"Top level directory where operation logs are stored if logging
functionality is enabled"),
{noformat}
Unless specified, both `SessionManager` instances use the default location,
which is same for both (both are running with the same user). On lens restart,
lens’s `SessionManager` cleans the directory. Hence the errors.
The simple solution would be to either run them as separate users, or on
separate machines.
For a different solution, we observe that operation logs are a freebie obtained
by lens when creating an instance of CLIService. There is no need for lens to
have operation logs, as no operations are created and no heavy lifting is done.
So disabling operation logs in lensserver-defaults.xml
> HiveDriver should create sessions independent of the transport
> --------------------------------------------------------------
>
> Key: LENS-1150
> URL: https://issues.apache.org/jira/browse/LENS-1150
> Project: Apache Lens
> Issue Type: Task
> Reporter: Rajat Khandelwal
> Assignee: Rajat Khandelwal
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
