[
https://issues.apache.org/jira/browse/HIVE-26374?focusedWorklogId=790584&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-790584
]
ASF GitHub Bot logged work on HIVE-26374:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 13/Jul/22 19:50
Start Date: 13/Jul/22 19:50
Worklog Time Spent: 10m
Work Description: cravani commented on code in PR #3419:
URL: https://github.com/apache/hive/pull/3419#discussion_r920434176
##########
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactionQueryBuilder.java:
##########
@@ -419,15 +421,13 @@ private void defineColumns(StringBuilder query) {
}
List<FieldSchema> cols = sourceTab.getSd().getCols();
boolean isFirst = true;
+ List<String> columnDescs = new ArrayList<String>();
Review Comment:
Fixed the code.
##########
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactionQueryBuilder.java:
##########
@@ -419,15 +421,13 @@ private void defineColumns(StringBuilder query) {
}
List<FieldSchema> cols = sourceTab.getSd().getCols();
boolean isFirst = true;
Review Comment:
Removed!
##########
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactionQueryBuilder.java:
##########
@@ -419,15 +421,13 @@ private void defineColumns(StringBuilder query) {
}
List<FieldSchema> cols = sourceTab.getSd().getCols();
boolean isFirst = true;
+ List<String> columnDescs = new ArrayList<String>();
for (FieldSchema col : cols) {
- if (!isFirst) {
- query.append(", ");
- }
- isFirst = false;
- query.append("`").append(col.getName()).append("` ");
- query.append(crud ? ":" : "");
- query.append(col.getType());
+ String columnType =
DDLPlanUtils.formatType(TypeInfoUtils.getTypeInfoFromTypeString(col.getType()));
+ String columnDesc = "`" + col.getName() + "`" + (crud ? ":" : "") +
columnType;
Review Comment:
We can add the extra space, honestly it won't make much of difference
because this would be a temporary table and would be dropped after compaction.
Issue Time Tracking
-------------------
Worklog Id: (was: 790584)
Time Spent: 1h (was: 50m)
> Query based compaction fails for tables with CDT and columns with Reserved
> Keywords
> -----------------------------------------------------------------------------------
>
> Key: HIVE-26374
> URL: https://issues.apache.org/jira/browse/HIVE-26374
> Project: Hive
> Issue Type: Bug
> Components: Hive, Transactions
> Affects Versions: 3.1.3, 4.0.0-alpha-1
> Reporter: Chiran Ravani
> Assignee: Chiran Ravani
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Query based compaction fails on Tables having complex data types with
> reserved keywords for columns. The compaction fails while creating a
> temporary table as it does not quote the columns correctly.
>
> Below are the steps to reproduce the issue.
> {code:java}
> create table complex_dt_compact2(col1 array<struct<arr_col1:int,
> `timestamp`:string>>);
> insert into complex_dt_compact2 SELECT
> ARRAY(NAMED_STRUCT('arr_col1',1,'timestamp','2022-07-05 21:51:20.371'));
> insert into complex_dt_compact2 SELECT
> ARRAY(NAMED_STRUCT('arr_col1',2,'timestamp','2022-07-05 21:51:20.371'));
> alter table complex_dt_compact2 compact 'major' and wait; {code}
> Error:
> {code:java}
> 2022-07-05T22:15:47.710Z
> hiveserver2-0.hiveserver2-service.compute-1657056457-xkcx.svc.cluster.local
> hiveserver2 1 dbb4011d-c788-4b99-a31d-06bb6dd7182e [mdc@18060
> class="compactor.Worker" level="ERROR"
> thread="hiveserver2-0.hiveserver2-service.compute-1657056457-xkcx.svc.cluster.local-64_executor"]
> Caught exception while trying to compact
> id:3,dbname:default,tableName:complex_dt_compact2,partName:null,state:,type:MAJOR,enqueueTime:0,start:0,properties:null,runAs:hive,tooManyAborts:false,hasOldAbort:false,highestWriteId:3,errorMessage:null,workerId:
> null,initiatorId: null,retryRetention0. Marking failed to avoid repeated
> failures
> java.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException:
> Failed to run CREATE temporary external table
> default_tmp_compactor_complex_dt_compact2_1657059347578(`operation` int,
> `originalTransaction` bigint, `bucket` int, `rowId` bigint,
> `currentTransaction` bigint, `row` struct<`col1`
> :array<struct<arr_col1:int,timestamp:string>>>) stored as orc LOCATION
> 's3a://obfuscated/clusters/obfuscated/obfuscated/warehouse/tablespace/managed/hive/complex_dt_compact2/base_0000003_v0000038'
> TBLPROPERTIES ('compactiontable'='true', 'transactional'='false')
> at
> org.apache.hadoop.hive.ql.txn.compactor.QueryCompactor.runCompactionQueries(QueryCompactor.java:120)
> at
> org.apache.hadoop.hive.ql.txn.compactor.MajorQueryCompactor.runCompaction(MajorQueryCompactor.java:63)
> at
> org.apache.hadoop.hive.ql.txn.compactor.Worker.findNextCompactionAndExecute(Worker.java:517)
> at
> org.apache.hadoop.hive.ql.txn.compactor.Worker.lambda$run$0(Worker.java:120)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Failed to
> run CREATE temporary external table
> default_tmp_compactor_complex_dt_compact2_1657059347578(`operation` int,
> `originalTransaction` bigint, `bucket` int, `rowId` bigint,
> `currentTransaction` bigint, `row` struct<`col1`
> :array<struct<arr_col1:int,timestamp:string>>>) stored as orc LOCATION
> 's3a://obfuscated/clusters/obfuscated/obfuscated/warehouse/tablespace/managed/hive/complex_dt_compact2/base_0000003_v0000038'
> TBLPROPERTIES ('compactiontable'='true', 'transactional'='false')
> at
> org.apache.hadoop.hive.ql.DriverUtils.runOnDriver(DriverUtils.java:73)
> at
> org.apache.hadoop.hive.ql.DriverUtils.runOnDriver(DriverUtils.java:50)
> at
> org.apache.hadoop.hive.ql.txn.compactor.QueryCompactor.runCompactionQueries(QueryCompactor.java:113)
> ... 7 more
> Caused by: (responseCode = 40000, errorMessage = FAILED: ParseException
> line 1:241 cannot recognize input near 'timestamp' ':' 'string' in column
> specification, SQLState = 42000, exception = line 1:241 cannot recognize
> input near 'timestamp' ':' 'string' in column specification)
> at
> org.apache.hadoop.hive.ql.DriverUtils.createProcessorException(DriverUtils.java:143)
> at
> org.apache.hadoop.hive.ql.Compiler.handleException(Compiler.java:466)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:122)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:197)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:636)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:694)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:526)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:515)
> at
> org.apache.hadoop.hive.ql.DriverUtils.runOnDriver(DriverUtils.java:70)
> ... 9 more{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
