[
https://issues.apache.org/jira/browse/HIVE-25413?focusedWorklogId=638486&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-638486
]
ASF GitHub Bot logged work on HIVE-25413:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 17/Aug/21 03:18
Start Date: 17/Aug/21 03:18
Worklog Time Spent: 10m
Work Description: rbalamohan commented on a change in pull request #2578:
URL: https://github.com/apache/hive/pull/2578#discussion_r690003539
##########
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java
##########
@@ -5096,37 +5097,74 @@ public Partition alterPartition(String catName, String
dbname, String name, List
}
@Override
- public List<Partition> alterPartitions(String catName, String dbname, String
name,
+ public List<Partition> alterPartitions(String catName, String dbName, String
tblName,
List<List<String>> part_vals, List<Partition>
newParts,
long writeId, String queryWriteIdList)
throws InvalidObjectException, MetaException
{
boolean success = false;
Exception e = null;
List<Partition> results = new ArrayList<>(newParts.size());
+ if (newParts.isEmpty()) {
+ return results;
+ }
try {
openTransaction();
- Iterator<List<String>> part_val_itr = part_vals.iterator();
- Set<MColumnDescriptor> oldCds = new HashSet<>();
- Ref<MColumnDescriptor> oldCdRef = new Ref<>();
- MTable table = null;
- for (Partition tmpPart: newParts) {
- List<String> tmpPartVals = part_val_itr.next();
- if (writeId > 0) {
- tmpPart.setWriteId(writeId);
+
+ MTable table = this.getMTable(catName, dbName, tblName);
+ List<String> partNames = new ArrayList<>();
+ for (List<String> partVal : part_vals) {
+ partNames.add(
+
Warehouse.makePartName(convertToFieldSchemas(table.getPartitionKeys()), partVal)
+ );
+ }
+
+ catName = normalizeIdentifier(catName);
+ dbName = normalizeIdentifier(dbName);
+ tblName = normalizeIdentifier(tblName);
+ List<MPartition> mPartitionList;
+
+ try (Query query = pm.newQuery(MPartition.class,
+ "table.tableName == t1 && table.database.name == t2 &&
t3.contains(partitionName) " +
+ " && table.database.catalogName == t4")) {
+ query.declareParameters("java.lang.String t1, java.lang.String t2,
java.util.Collection t3, "
+ + "java.lang.String t4");
+ mPartitionList = (List<MPartition>) query.executeWithArray(tblName,
dbName, partNames, catName);
Review comment:
This should help a lot in reducing calls when large number of partitions
are present.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 638486)
Time Spent: 20m (was: 10m)
> Optimise ObjectStore::alterPartitions to reduce DB calls for getPartitions
> --------------------------------------------------------------------------
>
> Key: HIVE-25413
> URL: https://issues.apache.org/jira/browse/HIVE-25413
> Project: Hive
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L5121]
> For dynamic load partitions (e.g when 1000s of parts being updated), it takes
> lot of time to fetch one partition at a time. It would be good to have
> "List<MPartition> getMPartitions" to reduce number of interactions with DB.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
