[
https://issues.apache.org/jira/browse/HIVE-26414?focusedWorklogId=796944&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796944
]
ASF GitHub Bot logged work on HIVE-26414:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 01/Aug/22 15:55
Start Date: 01/Aug/22 15:55
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #3457:
URL: https://github.com/apache/hive/pull/3457#discussion_r934683229
##########
ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java:
##########
@@ -3080,6 +3084,83 @@ public void testAcidOrcWritePreservesFieldNames() throws
Exception {
hiveConf.setBoolVar(HiveConf.ConfVars.HIVE_VECTORIZATION_ENABLED, true);
}
+ @Test
+ public void testFailureScenariosCleanupCTAS() throws Exception {
+ boolean[] booleans = {true, false};
+ for (boolean var1 : booleans) {
+ for (boolean var2 : booleans) {
+ for (boolean var3 : booleans) {
+ for (boolean var4 : booleans) {
+ failureScenarioCleanupCTAS(var1, var2, var3, var4);
+ }
+ }
+ }
+ }
+ }
+
+ public void failureScenarioCleanupCTAS(boolean isPartitioned,
+ boolean isDirectInsertEnabled,
+ boolean isLocklessReadsEnabled,
+ boolean isLocationUsed) throws
Exception {
+ String tableName = "atable";
+
+ //Set configurations
+ hiveConf.setBoolVar(HiveConf.ConfVars.HIVE_ACID_DIRECT_INSERT_ENABLED,
isDirectInsertEnabled);
+ hiveConf.setBoolVar(HiveConf.ConfVars.HIVE_ACID_LOCKLESS_READS_ENABLED,
isLocklessReadsEnabled);
+ hiveConf.setBoolVar(HiveConf.ConfVars.TXN_CTAS_X_LOCK, true);
+ hiveConf.setIntVar(HiveConf.ConfVars.HIVE_COMPACTOR_ABORTEDTXN_THRESHOLD,
0);
+
+ // Add a '1' at the end of table name for custom location.
+ String querylocation = (isLocationUsed) ? " location '" +
getWarehouseDir() + "/" + tableName + "1'" : "";
+ String queryPartitions = (isPartitioned) ? " partitioned by (a)" : "";
+
+ d.run("insert into " + Table.ACIDTBL + "(a,b) values (3,4)");
+ d.run("drop table if exists " + tableName);
+ d.compileAndRespond("create table " + tableName + queryPartitions + "
stored as orc" + querylocation +
+ " tblproperties ('transactional'='true') as select * from " +
Table.ACIDTBL);
+ long txnId = d.getQueryState().getTxnManager().getCurrentTxnId();
+ DriverContext driverContext = d.getDriverContext();
+ traverseTasksRecursively(driverContext.getPlan().getRootTasks());
Review Comment:
could be simplified with
````
MapRedTask mrtask = Mockito.spy(new MapRedTask());
Mockito.doThrow(new RuntimeException()).when(mrtask).execute();
driverContext.getPlan().setRootTasks(Lists.newArrayList(mrtask));
````
Issue Time Tracking
-------------------
Worklog Id: (was: 796944)
Time Spent: 11.5h (was: 11h 20m)
> Aborted/Cancelled CTAS operations must initiate cleanup of uncommitted data
> ---------------------------------------------------------------------------
>
> Key: HIVE-26414
> URL: https://issues.apache.org/jira/browse/HIVE-26414
> Project: Hive
> Issue Type: Improvement
> Reporter: Sourabh Badhya
> Assignee: Sourabh Badhya
> Priority: Major
> Labels: pull-request-available
> Time Spent: 11.5h
> Remaining Estimate: 0h
>
> When a CTAS query fails before creation of table and after writing the data,
> the data is present in the directory and not cleaned up currently by the
> cleaner or any other mechanism currently. This is because the cleaner
> requires a table corresponding to what its cleaning. In order surpass such a
> situation, we can directly pass the relevant information to the cleaner so
> that such uncommitted data is deleted.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
