[
https://issues.apache.org/jira/browse/HIVE-26482?focusedWorklogId=802824&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-802824
]
ASF GitHub Bot logged work on HIVE-26482:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 23/Aug/22 12:24
Start Date: 23/Aug/22 12:24
Worklog Time Spent: 10m
Work Description: InvisibleProgrammer commented on code in PR #3532:
URL: https://github.com/apache/hive/pull/3532#discussion_r952547957
##########
ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java:
##########
@@ -3264,6 +3264,37 @@ public void testNoTxnComponentsForScheduledQueries()
throws Exception {
Assert.assertEquals(resData, stringifyValues(actualData));
}
+ @Test
+ public void testCompactionOutputDirectoryNamesOnPartitions() throws
Exception {
+ String p1 = "p=p1";
+ String p2 = "p=p2";
+ String expectedDelta1 = p1 + "/delta_0000001_0000002_v0000021";
+ String expectedDelta2 = p2 + "/delta_0000003_0000004_v0000022";
+
+ runStatementOnDriver("insert into " + Table.ACIDTBLPART + "
partition(p='p1') (a,b) values(1,2)");
+ runStatementOnDriver("insert into " + Table.ACIDTBLPART + "
partition(p='p1') (a,b) values(3,4)");
+ runStatementOnDriver("insert into " + Table.ACIDTBLPART + "
partition(p='p2') (a,b) values(1,2)");
+ runStatementOnDriver("insert into " + Table.ACIDTBLPART + "
partition(p='p2') (a,b) values(3,4)");
+
+ compactPartition(Table.ACIDTBLPART.name().toLowerCase(),
CompactionType.MINOR, p1);
+ compactPartition(Table.ACIDTBLPART.name().toLowerCase(),
CompactionType.MINOR, p2);
+
+ FileSystem fs = FileSystem.get(hiveConf);
+ String tablePath = getWarehouseDir() + "/" +
Table.ACIDTBLPART.name().toLowerCase() + "/";
+
+ Assert.assertTrue(fs.exists(new Path(tablePath + expectedDelta1)));
+ Assert.assertTrue(fs.exists(new Path(tablePath + expectedDelta2)));
Review Comment:
Extra asserts added.
Issue Time Tracking
-------------------
Worklog Id: (was: 802824)
Time Spent: 50m (was: 40m)
> Create a unit test checking compaction output file names on a partitioned
> table
> -------------------------------------------------------------------------------
>
> Key: HIVE-26482
> URL: https://issues.apache.org/jira/browse/HIVE-26482
> Project: Hive
> Issue Type: Test
> Components: Hive
> Reporter: Zsolt Miskolczi
> Assignee: Zsolt Miskolczi
> Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Compaction output directories' writeIds only reflect the writeIds of the
> deltas it compacts, and not the max write id of the table
> Example:
> Pre-compaction...
> {code:java}
> Partition p=1 contains:
> delta_1_1
> delta_2_2
> partition p=2 contains
> delta_3_3
> delta_4_4
> {code}
> After minor compaction...
> {code:java}
> Partition p=1 contains:
> delta_1_2
> partition p=2 contains
> delta_3_4
> {code}
> AFAIK there are no unit tests that reflect this.
> TestTxnCommands2#testFullACIDAbortWithManyPartitions is a good template to
> start with.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
