[
https://issues.apache.org/jira/browse/HIVE-27266?focusedWorklogId=857200&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857200
]
ASF GitHub Bot logged work on HIVE-27266:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 15/Apr/23 02:06
Start Date: 15/Apr/23 02:06
Worklog Time Spent: 10m
Work Description: wecharyu commented on code in PR #4238:
URL: https://github.com/apache/hive/pull/4238#discussion_r1167357919
##########
standalone-metastore/metastore-tools/metastore-benchmarks/src/main/java/org/apache/hadoop/hive/metastore/tools/HMSBenchmarks.java:
##########
@@ -123,6 +123,31 @@ static DescriptiveStatistics
benchmarkDeleteWithPartitions(@NotNull MicroBenchma
null);
}
+ static DescriptiveStatistics benchmarkDeleteMetaOnlyWithPartitions(@NotNull
MicroBenchmark bench,
+
@NotNull BenchData data,
+ int
howMany,
+ int
nparams) {
+ final HMSClient client = data.getClient();
+ String dbName = data.dbName;
+ String tableName = data.tableName;
+
+ // Create many parameters
+ Map<String, String> parameters = new HashMap<>(nparams);
+ for (int i = 0; i < nparams; i++) {
+ parameters.put(PARAM_KEY + i, PARAM_VALUE + i);
+ }
+
+ return bench.measure(
+ () -> throwingSupplierWrapper(() -> {
+ BenchmarkUtils.createPartitionedTable(client, dbName, tableName);
+ addManyPartitions(client, dbName, tableName, parameters,
+ Collections.singletonList("d"), howMany);
Review Comment:
It's a prefix of the partition value, in this benchmark the created
partitions in mysql looks as follows:
```sql
mysql> select * from PARTITIONS limit 3;
+---------+-------------+------------------+-----------+--------+--------+----------+
| PART_ID | CREATE_TIME | LAST_ACCESS_TIME | PART_NAME | SD_ID | TBL_ID |
WRITE_ID |
+---------+-------------+------------------+-----------+--------+--------+----------+
| 301296 | 1681524241 | 0 | date=d0 | 302595 | 1299 |
0 |
| 301297 | 1681524241 | 0 | date=d1 | 302596 | 1299 |
0 |
| 301298 | 1681524241 | 0 | date=d2 | 302597 | 1299 |
0 |
+---------+-------------+------------------+-----------+--------+--------+----------+
3 rows in set (0.00 sec)
```
Issue Time Tracking
-------------------
Worklog Id: (was: 857200)
Time Spent: 40m (was: 0.5h)
> Retrieve only partNames if not need drop data in
> HMSHandler.dropPartitionsAndGetLocations
> -----------------------------------------------------------------------------------------
>
> Key: HIVE-27266
> URL: https://issues.apache.org/jira/browse/HIVE-27266
> Project: Hive
> Issue Type: Improvement
> Components: Hive
> Affects Versions: 4.0.0-alpha-2
> Reporter: Wechar
> Assignee: Wechar
> Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Followed HIVE-19783, we only need partNames instead of partName and location
> pairs if we do not need check location.
> We add a new benchmark *dropTableMetadataWithPartitions* to delete only
> metadata rather than the real table data.
> Test results like:
> * Before the patch:
> {code:bash}
> Operation Mean Med Min Max Err%
> dropTableMetaOnlyWithPartitions.10 23.70 21.87 19.36 31.73 14.48
> dropTableMetaOnlyWithPartitions.100 54.42 54.15 45.92 76.68 8.891
> dropTableMetaOnlyWithPartitions.1000 462.5 456.1 321.0 654.3 15.96
> {code}
> * After the patch:
> {code:bash}
> Operation Mean Med Min Max Err%
> dropTableMetaOnlyWithPartitions.10 21.49 21.24 19.30 27.90 6.661
> dropTableMetaOnlyWithPartitions.100 51.51 48.30 44.86 85.23 16.91
> dropTableMetaOnlyWithPartitions.1000 415.4 407.2 308.8 595.2 14.28
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
