Vihang Karajgaonkar has uploaded a new patch set (#3). (
http://gerrit.cloudera.org:8080/14750 )
Change subject: IMPALA-9092: Add support for creating external Kudu table
......................................................................
IMPALA-9092: Add support for creating external Kudu table
In HMS-3 the translation layer converts a managed kudu table into a external
kudu table
and adds additional table property 'external.table.purge' to 'true'. This means
any
installation which is using HMS-3 (or a Hive version which has HIVE-22158) will
always
create Kudu tables as external tables. This is problematic since the the output
of show
create table will now be different and may confuse the users.
In order to improve the user experience of such synchronized tables (external
tables with
external.table.purge property set to true), this patch adds support in Impala
to create
external Kudu tables. Previous versions of Impala disallowed creating a
external Kudu
table if the Kudu table did not exist. After this patch, Impala will check if
the Kudu
table exists and if it does not it will create a Kudu table based on the schema
provided
in the create table statement. The command will error out if the Kudu table
already
exists. However, this applies to only the synchronized tables. Previous way to
create a
pure external table behaves the same.
Following syntax of creating a synchronized table is now allowed:
CREATE EXTERNAL TABLE foo (
id int PRIMARY KEY,
name string)
PARTITION BY HASH PARTITIONS 8
STORED AS KUDU
TBLPROPERTIES ('external.table.purge'='true')
The syntax is very similar to creating a managed table, except for the EXTERNAL
keyword
and additional table property. A synchronized table will behave similar to
managed Kudu
tables (drops and renames are allowed). The output of show create table on a
synchronized
table will display the full column and partition spec similar to the managed
tables.
Testing:
1. After the CDP version bump all of the existing Kudu tables now create
synchronized
tables so there is good coverage there.
2. Added additional tests which create synchronized tables and compares the
show create
table output.
Change-Id: I76f81d41db0cf2269ee1b365857164a43677e14d
---
M fe/src/main/java/org/apache/impala/analysis/AlterTableSetTblProperties.java
M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java
M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java
M fe/src/main/java/org/apache/impala/catalog/KuduTable.java
M fe/src/main/java/org/apache/impala/catalog/Table.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/service/MetadataOp.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java
M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java
M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java
M fe/src/test/java/org/apache/impala/common/FrontendFixture.java
M testdata/workloads/functional-query/queries/QueryTest/show-create-table.test
M tests/common/skip.py
M tests/metadata/test_show_create_table.py
M tests/query_test/test_kudu.py
16 files changed, 748 insertions(+), 262 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/50/14750/3
--
To view, visit http://gerrit.cloudera.org:8080/14750
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I76f81d41db0cf2269ee1b365857164a43677e14d
Gerrit-Change-Number: 14750
Gerrit-PatchSet: 3
Gerrit-Owner: Vihang Karajgaonkar <[email protected]>
Gerrit-Reviewer: Andrew Wong <[email protected]>
Gerrit-Reviewer: Grant Henke <[email protected]>
Gerrit-Reviewer: Hao Hao <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]>
