Hello Tidy Bot, Kudu Jenkins, Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/8312
to look at the new patch set (#3).
Change subject: KUDU-2191 (4/n): WIP: Hive Metastore catalog manager integration
......................................................................
KUDU-2191 (4/n): WIP: Hive Metastore catalog manager integration
This commit adds a basic integration with the Hive Metastore, which can
be enabled with a new flag: 'hive-metastore-addresses'. When the flag is
provided, the Master will push DDL changes in its catalog to the Hive
Metastore. In particular:
- Create table, delete table, and alter table operations in the Kudu
master will synchronously update the corresponding entry in the HMS.
- New table and column names are required to be valid according to the
Hive identifier rules, which are much stricter than Kudu's native
identifier rules.
WIP: This patch doesn't yet have sufficient error handling logic, nor
test coverage of errors. Here are the test cases I intend to add to
master_hms-itest before this is committed:
- Rename a Kudu table to a new name which already has an entry in the HMS
(should fail).
- Rename a Kudu table when the original name doesn't have an entry in
the HMS (should create an entry with the new name).
- Rename a Kudu table when the HMS is unreachable (should fail).
- Create a Kudu table with a name which already exists in the HMS
(should fail).
- Create a Kudu table when the HMS is unreachable (should fail).
- Drop a Kudu table when the corresponding entry in the HMS does not
exist (should succeed).
- Drop a Kudu table when the HMS is unreachable (should fail?).
Additionally, this patch as written overwrites the entire HMS table
entry during table rename operations. This is bad because it will wipe
out any stats that query engines like Impala have attached to the table.
Change-Id: I1b8f360408b05d6ac4f7359a8ac0dd889b196baf
---
M src/kudu/integration-tests/CMakeLists.txt
A src/kudu/integration-tests/master_hms-itest.cc
M src/kudu/master/CMakeLists.txt
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
A src/kudu/master/hms_catalog.cc
A src/kudu/master/hms_catalog.h
M src/kudu/master/master.proto
M src/kudu/mini-cluster/external_mini_cluster.cc
M src/kudu/util/net/net_util.cc
M src/kudu/util/net/net_util.h
11 files changed, 649 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/12/8312/3
--
To view, visit http://gerrit.cloudera.org:8080/8312
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1b8f360408b05d6ac4f7359a8ac0dd889b196baf
Gerrit-Change-Number: 8312
Gerrit-PatchSet: 3
Gerrit-Owner: Dan Burkert <[email protected]>
Gerrit-Reviewer: Adar Dembo <[email protected]>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
