Amareshwari Sriramadasu created LENS-1165:
---------------------------------------------
Summary: Separate client for query submission from other thrift
operations
Key: LENS-1165
URL: https://issues.apache.org/jira/browse/LENS-1165
Project: Apache Lens
Issue Type: Improvement
Components: driver-hive
Reporter: Amareshwari Sriramadasu
Assignee: Amareshwari Sriramadasu
Fix For: 2.6
Proposing to add separate thrift client for query submissions in HiveDriver
with ability to give different thrift read timeouts and number of retries.
This would help in the following cases :
We have seen instances of query submission which can take more time. For ex :
----
2016-06-02T20:20:21,436 INFO [ef431743-b9f8-4fe0-8e3d-2bab154438eb
HiveServer2-Handler-Pool: Thread-66837]: ql.Driver (Driver.java:compile(372)) -
Compiling
command(queryId=hive_20160602195701_2a3bd795-0919-4e2e-ad22-812881b084fd):
2016-06-02T21:02:43,380 INFO [ef431743-b9f8-4fe0-8e3d-2bab154438eb
HiveServer2-Handler-Pool: Thread-66255]: ql.Driver (Driver.java:compile(524)) -
Completed compiling
command(queryId=hive_20160602200002_e58e07f9-61d0-423d-86ef-a626eb250114); Time
taken: 1830.214 seconds
---
In such cases lens submission will timeout with default client and resubmit,
which would take more time again.
And eventually, there will be number of submissions equal to the number of
retries configured and lens would have marked the query as failed. But
submission would actually launch multiple queries.
If we have separate client, it can be configured with different timeout value
and number retries to be different.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
