Casey Ching has uploaded a new change for review.
http://gerrit.cloudera.org:8080/2617
Change subject: Simplify creating external Kudu tables
......................................................................
Simplify creating external Kudu tables
(I'm still working on getting the python Kudu client integrated into the
virtualenv, ignore that for now.)
Creating an external Kudu table was a lot harder than it needed to be
because the columns needed to be specified twice, once in Kudu and once
again in Impala. Also table properties needed to be specified and that
is more tedious than it needs to be.
Changes:
1) Read table schema from Kudu. When attempting to create an external
table "foo" in database "bar", Impala will search for a Kudu table
name "foo.bar" and "bar" (Kudu doesn't have database name spaces
yet.)
2) The Kudu table is now required to exist at the time of creation in
Impala.
3) Disallow table properties that could conflict with an existing
table. Ex: key_columns cannot be specified.
4) Add KUDU as a file format.
5) Add a startup flag to catalogd to specify the default Kudu master
addresses. The flag is used as the default value for the table
property kudu_master_addresses.
Now an external Kudu table can be created with "CREATE EXTERNAL TABLE t
STORED AS KUDU" assuming table "t" exists in Kudu. Users can still
override table properties such as the Kudu table name or master
addresses in the usual way.
Change-Id: Ic141102818b6dad3016181b179a14024d0ff709d
---
M be/src/catalog/catalog.cc
M bin/start-catalogd.sh
M bin/start-impala-cluster.py
M common/thrift/CatalogObjects.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/com/cloudera/impala/analysis/ColumnDef.java
M fe/src/main/java/com/cloudera/impala/analysis/CreateTableDataSrcStmt.java
M fe/src/main/java/com/cloudera/impala/analysis/CreateTableStmt.java
M fe/src/main/java/com/cloudera/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/com/cloudera/impala/catalog/HdfsFileFormat.java
M fe/src/main/java/com/cloudera/impala/catalog/delegates/KuduDdlDelegate.java
M fe/src/main/java/com/cloudera/impala/service/CatalogOpExecutor.java
M fe/src/main/java/com/cloudera/impala/service/JniCatalog.java
A fe/src/main/java/com/cloudera/impala/util/KuduClient.java
M fe/src/main/java/com/cloudera/impala/util/KuduUtil.java
M fe/src/main/jflex/sql-scanner.flex
M fe/src/test/java/com/cloudera/impala/analysis/AnalyzeDDLTest.java
M fe/src/test/java/com/cloudera/impala/analysis/ParserTest.java
M fe/src/test/java/com/cloudera/impala/testutil/CatalogServiceTestCatalog.java
M testdata/bin/generate-schema-statements.py
M testdata/datasets/functional/functional_schema_template.sql
M testdata/workloads/functional-query/queries/QueryTest/create_kudu.test
M testdata/workloads/functional-query/queries/QueryTest/kudu-scan-node.test
M testdata/workloads/functional-query/queries/QueryTest/kudu-show-create.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_crud.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_partition_ddl.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_stats.test
M tests/common/__init__.py
A tests/common/kudu_test_suite.py
M tests/conftest.py
A tests/custom_cluster/test_kudu.py
M tests/query_test/test_kudu.py
33 files changed, 1,152 insertions(+), 433 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/17/2617/1
--
To view, visit http://gerrit.cloudera.org:8080/2617
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic141102818b6dad3016181b179a14024d0ff709d
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Casey Ching <[email protected]>
