[
https://issues.apache.org/jira/browse/HIVE-27150?focusedWorklogId=854207&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854207
]
ASF GitHub Bot logged work on HIVE-27150:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 31/Mar/23 14:47
Start Date: 31/Mar/23 14:47
Worklog Time Spent: 10m
Work Description: wecharyu commented on code in PR #4123:
URL: https://github.com/apache/hive/pull/4123#discussion_r1154570802
##########
standalone-metastore/metastore-tools/metastore-benchmarks/src/main/java/org/apache/hadoop/hive/metastore/tools/HMSBenchmarks.java:
##########
@@ -246,26 +246,24 @@ static DescriptiveStatistics
benchmarkGetPartitions(@NotNull MicroBenchmark benc
}
static DescriptiveStatistics benchmarkDropPartition(@NotNull MicroBenchmark
bench,
- @NotNull BenchData data)
{
+ @NotNull BenchData data,
+ int count) {
final HMSClient client = data.getClient();
String dbName = data.dbName;
String tableName = data.tableName;
BenchmarkUtils.createPartitionedTable(client, dbName, tableName);
- final List<String> values = Collections.singletonList("d1");
try {
- Table t = client.getTable(dbName, tableName);
- Partition partition = new Util.PartitionBuilder(t)
- .withValues(values)
- .build();
-
return bench.measure(
- () -> throwingSupplierWrapper(() -> client.addPartition(partition)),
- () -> throwingSupplierWrapper(() -> client.dropPartition(dbName,
tableName, values)),
+ () -> addManyPartitionsNoException(client, dbName, tableName, null,
+ Collections.singletonList("d"), count),
+ () -> throwingSupplierWrapper(() -> {
+ List<String> partNames = client.getPartitionNames(dbName,
tableName);
+ partNames.forEach(partName ->
+ throwingSupplierWrapper(() -> client.dropPartition(dbName,
tableName, partName)));
+ return null;
+ }),
null);
- } catch (TException e) {
Review Comment:
I followed the `benchmarkDropPartitions()` and some others, catching
exception here should have nothing to do with benchmark statistics, because the
result statistic is in `measure()` method:
https://github.com/apache/hive/blob/745dbf7059de93a74bceda5e5acbb293008771dc/standalone-metastore/metastore-tools/tools-common/src/main/java/org/apache/hadoop/hive/metastore/tools/MicroBenchmark.java#L84-L87
Also the `Err%` should be **Coefficient of variation** of successful
operations rather than percentage of error records:
https://github.com/apache/hive/blob/745dbf7059de93a74bceda5e5acbb293008771dc/standalone-metastore/metastore-tools/tools-common/src/main/java/org/apache/hadoop/hive/metastore/tools/BenchmarkSuite.java#L207
Issue Time Tracking
-------------------
Worklog Id: (was: 854207)
Time Spent: 4h (was: 3h 50m)
> Drop single partition can also support direct sql
> -------------------------------------------------
>
> Key: HIVE-27150
> URL: https://issues.apache.org/jira/browse/HIVE-27150
> Project: Hive
> Issue Type: Improvement
> Components: Hive
> Reporter: Wechar
> Assignee: Wechar
> Priority: Major
> Labels: pull-request-available
> Time Spent: 4h
> Remaining Estimate: 0h
>
> *Background:*
> [HIVE-6980|https://issues.apache.org/jira/browse/HIVE-6980] supports direct
> sql for drop_partitions, we can reuse this huge improvement in drop_partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
