Gowthami Bisati has uploaded a new patch set (#4). (
http://gerrit.cloudera.org:8080/24609 )
Change subject: IMPALA-10693: Allow TBLPROPERTIES in CREATE TABLE LIKE
......................................................................
IMPALA-10693: Allow TBLPROPERTIES in CREATE TABLE LIKE
Currently, the CREATE TABLE LIKE statement in Impala copies the source
table's metadata (including TBLPROPERTIES) exactly as-is. There is no
way to override or add new table properties during creation.
This patch adds support for an optional TBLPROPERTIES clause to the
CREATE TABLE LIKE syntax. User-specified properties are merged with or
override the properties inherited from the source table.
Change Summary:
1. sql-parser.cup: Add optional TBLPROPERTIES clause parsing rules for
CREATE TABLE LIKE statements.
2. CreateTableLikeStmt.java: Store the parsed properties map and forward
it to the thrift parameter container.
3. JniCatalog.thrift: Add optional map<string,string> tbl_properties to
TCreateTableLikeParams.
4. CatalogOpExecutor.java: Merge the new tbl_properties into the target
table parameters (handling Iceberg properties appropriately) before
persisting the table in the metastore.
5. Added AnalyzeDDLTest#TestCreateTableLikeProperties to verify frontend
analyzer behavior and property validation.
After the code change:
[localhost:21050] default> CREATE TABLE default.my_internal_table
> LIKE default.my_external_table
> TBLPROPERTIES (
'transactional'='true',
'transactional_properties'='insert_only');
Query: CREATE TABLE default.my_internal_table
LIKE default.my_external_table
TBLPROPERTIES (
'transactional'='true',
'transactional_properties'='insert_only')
+-------------------------+
| summary |
+-------------------------+
| Table has been created. |
+-------------------------+
Fetched 1 row(s) in 0.10s
[localhost:21050] default> show create table default.my_internal_table ;
Query: show create table default.my_internal_table
+----------------------------------------------------------------------------+
| result |
+----------------------------------------------------------------------------+
| CREATE TABLE default.my_internal_table ( |
| id INT, |
| name STRING, |
| created_date TIMESTAMP |
| ) |
| ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' |
| WITH SERDEPROPERTIES ( |
| 'field.delim'=',', |
| 'serialization.format'=',' |
| ) |
| STORED AS TEXTFILE |
| LOCATION 'hdfs://localhost:20500/test-warehouse/managed/my_internal_table' |
| TBLPROPERTIES ( |
| 'OBJCAPABILITIES'='EXTREAD,EXTWRITE', |
| 'transactional'='true', |
| 'transactional_properties'='insert_only' |
| ) |
+----------------------------------------------------------------------------+
2. This change allows users to override or add table properties
during a CREATE TABLE LIKE statement.
For example, specifying 'transactional'='true' now executes
successfully instead of throwing a grammar ParseException.
CREATE TABLE orders_acid LIKE orders
STORED AS ORC
TBLPROPERTIES ('transactional'='true')
ParseException: Syntax error in line 3:undefined: TBLPROPERTIES
('transactional'='true') ^ Encountered: TBLPROPERTIES Expected:
LOCATION CAUSED BY: Exception: Syntax error
After this change:
[localhost:21050] default> show create table source_tbl;
Query: show create table source_tbl
+-------------------------------------------------------------+
| result |
+-------------------------------------------------------------+
| CREATE EXTERNAL TABLE default.source_tbl ( |
| id INT |
| ) |
| STORED AS TEXTFILE |
| LOCATION 'hdfs://localhost:20500/test-warehouse/source_tbl' |
| TBLPROPERTIES ( |
| 'OBJCAPABILITIES'='EXTREAD,EXTWRITE', |
| 'TRANSLATED_TO_EXTERNAL'='TRUE', |
| 'env'='prod', |
| 'external.table.purge'='TRUE', |
| 'owner'='alice' |
| ) |
+-------------------------------------------------------------+
[localhost:21050] default> CREATE TABLE target_tbl LIKE source_tbl
> TBLPROPERTIES('env'='dev','team'='analytics');
Query: CREATE TABLE target_tbl LIKE source_tbl
TBLPROPERTIES('env'='dev','team'='analytics')
+-------------------------+
| summary |
+-------------------------+
| Table has been created. |
+-------------------------+
Fetched 1 row(s) in 0.10s
[localhost:21050] default> show create table target_tbl;
Query: show create table target_tbl
+-------------------------------------------------------------+
| result |
+-------------------------------------------------------------+
| CREATE EXTERNAL TABLE default.target_tbl ( |
| id INT |
| ) |
| STORED AS TEXTFILE |
| LOCATION 'hdfs://localhost:20500/test-warehouse/target_tbl' |
| TBLPROPERTIES ( |
| 'OBJCAPABILITIES'='EXTREAD,EXTWRITE', |
| 'TRANSLATED_TO_EXTERNAL'='TRUE', |
| 'env'='dev', |
| 'external.table.purge'='TRUE', |
| 'owner'='alice', |
| 'team'='analytics' |
| ) |
+-------------------------------------------------------------+
Tests Performed:
- Verified frontend changes compile cleanly using checkstyle.
- mvn clean package -DskipTests (BUILD SUCCESS).
- mvn test -Dtest=AnalyzeDDLTest#TestCreateTableLikeProperties (BUILD SUCCESS).
- mvn test -Dtest=ToSqlTest (BUILD SUCCESS).
- Ran idempotency integrity validation suite:
mvn test -Dtest=ToSqlUtilsTest (BUILD SUCCESS).
- python3 bin/jenkins/critique-gerrit-review.py --dryrun
- tests/run-tests.py query_test/test_kudu.py
Change-Id: I5d45bdfeebbc8107abd51f87145330dbbb675855
Signed-off-by: Gowthami Bisati <[email protected]>
---
M common/thrift/JniCatalog.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
5 files changed, 63 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/09/24609/4
--
To view, visit http://gerrit.cloudera.org:8080/24609
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5d45bdfeebbc8107abd51f87145330dbbb675855
Gerrit-Change-Number: 24609
Gerrit-PatchSet: 4
Gerrit-Owner: Gowthami Bisati <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
