[
https://issues.apache.org/jira/browse/HIVE-23949?focusedWorklogId=464579&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-464579
]
ASF GitHub Bot logged work on HIVE-23949:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 30/Jul/20 14:28
Start Date: 30/Jul/20 14:28
Worklog Time Spent: 10m
Work Description: soumyakanti3578 commented on a change in pull request
#1317:
URL: https://github.com/apache/hive/pull/1317#discussion_r463036109
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
##########
@@ -3665,33 +3678,39 @@ public boolean dropPartition(String dbName, String
tableName, List<String> parti
* @return list of partition objects
*/
public List<Partition> getPartitions(Table tbl) throws HiveException {
- if (tbl.isPartitioned()) {
- List<org.apache.hadoop.hive.metastore.api.Partition> tParts;
- try {
- GetPartitionsPsWithAuthRequest req = new
GetPartitionsPsWithAuthRequest();
- req.setTblName(tbl.getTableName());
- req.setDbName(tbl.getDbName());
- req.setUserName(getUserName());
- req.setMaxParts((short) -1);
- req.setGroupNames(getGroupNames());
- if (AcidUtils.isTransactionalTable(tbl)) {
- ValidWriteIdList validWriteIdList =
getValidWriteIdList(tbl.getDbName(), tbl.getTableName());
- req.setValidWriteIdList(validWriteIdList != null ?
validWriteIdList.toString() : null);
- }
- GetPartitionsPsWithAuthResponse res =
getMSC().listPartitionsWithAuthInfoRequest(req);
- tParts = res.getPartitions();
+ long t1 = System.nanoTime();
+ try {
+ if (tbl.isPartitioned()) {
+ List<org.apache.hadoop.hive.metastore.api.Partition> tParts;
+ try {
+ GetPartitionsPsWithAuthRequest req = new
GetPartitionsPsWithAuthRequest();
+ req.setTblName(tbl.getTableName());
+ req.setDbName(tbl.getDbName());
+ req.setUserName(getUserName());
+ req.setMaxParts((short) -1);
+ req.setGroupNames(getGroupNames());
+ if (AcidUtils.isTransactionalTable(tbl)) {
+ ValidWriteIdList validWriteIdList =
getValidWriteIdList(tbl.getDbName(), tbl.getTableName());
+ req.setValidWriteIdList(validWriteIdList != null ?
validWriteIdList.toString() : null);
+ }
+ GetPartitionsPsWithAuthResponse res =
getMSC().listPartitionsWithAuthInfoRequest(req);
+ tParts = res.getPartitions();
- } catch (Exception e) {
- LOG.error(StringUtils.stringifyException(e));
- throw new HiveException(e);
- }
- List<Partition> parts = new ArrayList<>(tParts.size());
- for (org.apache.hadoop.hive.metastore.api.Partition tpart : tParts) {
- parts.add(new Partition(tbl, tpart));
+ } catch (Exception e) {
+ LOG.error(StringUtils.stringifyException(e));
+ throw new HiveException(e);
+ }
+ List<Partition> parts = new ArrayList<>(tParts.size());
+ for (org.apache.hadoop.hive.metastore.api.Partition tpart : tParts) {
+ parts.add(new Partition(tbl, tpart));
+ }
+ return parts;
+ } else {
+ return Collections.singletonList(new Partition(tbl));
}
- return parts;
- } else {
- return Collections.singletonList(new Partition(tbl));
+ } finally {
Review comment:
Since there are many instances for 2., I am moving all `PerfLogEnd` to
`finally` block for now, which will probably be easier to maintain in the
future, because of uniformity.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 464579)
Time Spent: 40m (was: 0.5h)
> Introduce caching layer in HS2 to accelerate query compilation
> --------------------------------------------------------------
>
> Key: HIVE-23949
> URL: https://issues.apache.org/jira/browse/HIVE-23949
> Project: Hive
> Issue Type: New Feature
> Components: HiveServer2
> Reporter: Soumyakanti Das
> Assignee: Soumyakanti Das
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> One of the major contributors to compilation latency is the retrieval of
> metadata from HMS. This JIRA introduces a caching layer in HS2 for this
> metadata. Its design is simple, relying on snapshot information being queried
> to cache and invalidate the metadata. This will help us to reduce the time
> spent in compilation by using HS2 memory more effectively, and it will allow
> us to improve HMS throughput for multi-tenant workloads by reducing the
> number of calls it needs to serve.
> This patch only caches partition retrieval information, which is often one of
> the most costly metadata operations. It also sets the foundation to cache
> additional calls in follow-up work.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
