[
https://issues.apache.org/jira/browse/HIVE-26035?focusedWorklogId=840806&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-840806
]
ASF GitHub Bot logged work on HIVE-26035:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 21/Jan/23 15:14
Start Date: 21/Jan/23 15:14
Worklog Time Spent: 10m
Work Description: VenuReddy2103 commented on code in PR #3905:
URL: https://github.com/apache/hive/pull/3905#discussion_r1083300357
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java:
##########
@@ -2615,30 +2625,60 @@ public boolean addPartitions(String catName, String
dbName, String tblName, List
MPartition mpart = convertToMPart(part, table, true);
toPersist.add(mpart);
+ mParts.add(mpart);
int now = (int) (System.currentTimeMillis() / 1000);
+ List<MPartitionPrivilege> mPartPrivileges = new ArrayList<>();
if (tabGrants != null) {
for (MTablePrivilege tab: tabGrants) {
- toPersist.add(new MPartitionPrivilege(tab.getPrincipalName(),
- tab.getPrincipalType(), mpart, tab.getPrivilege(), now,
- tab.getGrantor(), tab.getGrantorType(), tab.getGrantOption(),
- tab.getAuthorizer()));
+ MPartitionPrivilege mPartPrivilege = new
MPartitionPrivilege(tab.getPrincipalName(), tab.getPrincipalType(),
+ mpart, tab.getPrivilege(), now, tab.getGrantor(),
tab.getGrantorType(), tab.getGrantOption(),
+ tab.getAuthorizer());
+ toPersist.add(mPartPrivilege);
+ mPartPrivileges.add(mPartPrivilege);
}
}
+ List<MPartitionColumnPrivilege> mPartColumnPrivileges = new
ArrayList<>();
if (tabColumnGrants != null) {
for (MTableColumnPrivilege col : tabColumnGrants) {
- toPersist.add(new MPartitionColumnPrivilege(col.getPrincipalName(),
- col.getPrincipalType(), mpart, col.getColumnName(),
col.getPrivilege(),
- now, col.getGrantor(), col.getGrantorType(),
col.getGrantOption(),
- col.getAuthorizer()));
+ MPartitionColumnPrivilege mPartColumnPrivilege = new
MPartitionColumnPrivilege(col.getPrincipalName(),
+ col.getPrincipalType(), mpart, col.getColumnName(),
col.getPrivilege(), now, col.getGrantor(),
+ col.getGrantorType(), col.getGrantOption(),
col.getAuthorizer());
+ toPersist.add(mPartColumnPrivilege);
+ mPartColumnPrivileges.add(mPartColumnPrivilege);
}
}
+ mPartPrivilegesList.add(mPartPrivileges);
+ mPartColPrivilegesList.add(mPartColumnPrivileges);
}
if (CollectionUtils.isNotEmpty(toPersist)) {
- pm.makePersistentAll(toPersist);
- pm.flush();
- }
+ GetHelper<Void> helper = new GetHelper<Void>(null, null, null, true,
+ true) {
+ @Override
+ protected Void getSqlResult(GetHelper<Void> ctx) throws
MetaException {
+ directSql.addPartitions(mParts, mPartPrivilegesList,
mPartColPrivilegesList);
+ return null;
+ }
+
+ @Override
+ protected Void getJdoResult(GetHelper<Void> ctx) {
+ reInstateTransaction();
Review Comment:
removed from here now
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java:
##########
@@ -2607,6 +2614,9 @@ public boolean addPartitions(String catName, String
dbName, String tblName, List
tabColumnGrants = this.listTableAllColumnGrants(catName, dbName,
tblName);
}
List<Object> toPersist = new ArrayList<>();
Review Comment:
done
Issue Time Tracking
-------------------
Worklog Id: (was: 840806)
Time Spent: 3h (was: 2h 50m)
> Explore moving to directsql for ObjectStore::addPartitions
> ----------------------------------------------------------
>
> Key: HIVE-26035
> URL: https://issues.apache.org/jira/browse/HIVE-26035
> Project: Hive
> Issue Type: Bug
> Reporter: Rajesh Balamohan
> Assignee: Venugopal Reddy K
> Priority: Major
> Labels: pull-request-available
> Time Spent: 3h
> Remaining Estimate: 0h
>
> Currently {{addPartitions}} uses datanuclues and is super slow for large
> number of partitions. It will be good to move to direct sql. Lots of repeated
> SQLs can be avoided as well (e.g SDS, SERDE, TABLE_PARAMS)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
