[
https://issues.apache.org/jira/browse/IMPALA-9030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16956698#comment-16956698
]
ASF subversion and git services commented on IMPALA-9030:
---------------------------------------------------------
Commit e05a5323785ecb09e45bdb5dfc96533e68256175 in impala's branch
refs/heads/master from Vihang Karajgaonkar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=e05a532 ]
IMPALA-9030: Handle translated external Kudu tables
In HMS 3.1 there is a default tranformer introduced which checks the client
capabilities
and transforms a table before creating it. Additionally, it also makes sure
that any
managed table which is created is transactional. If a user creates a managed
table which
is not transactional, it automatically converts such table as external and sets
certain
table properties to mark such transformed tables.
This presents a problem for managed Kudu tables in Impala since managed and
external
tables are handled differently in Kudu. Specifically, if a Kudu table is
managed, certain
operations like drop table, rename table, alter table are performed on the Kudu
side along
with updating the catalog. If the Kudu table is external, the Kudu operations
are skipped
and only catalog side operations are performed.
When the user creates a managed Kudu table, user expects that drop table,
rename table
should be updated by Impala automatically in Kudu as well. But since HMS 3
transforms such
managed tables into external, currently Impala does not perform the Kudu side
operations
breaking the semantics for the user.
This patch makes changes to Catalog so that it can detect such transformed
external tables
and perform Kudu side operations similar to what it was doing for managed Kudu
table when
talking with previous HMS versions.
Note that this change is in preparation of bumping up the CDP build which will
be done in
a separate change. For the current CDP build number the patch is essentially a
no-op.
Testing:
1. Bumped up the CDP build number in a private build so that the HMS
translation logic is pulled in. Ran all the tests. Without the patch there are
many Kudu
tests which were failing. After the patch none of the Kudu tests fail. There
were
additional Ranger tests which failed due to the CDP bump but those were
unrelated to
this patch and should be fixed as part of a separate change when the CDP build
number
is bumped up.
Change-Id: I324523361c923b7d291cb4d0f1028b1a5b653b36
Reviewed-on: http://gerrit.cloudera.org:8080/14397
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Handle translated external Kudu tables
> --------------------------------------
>
> Key: IMPALA-9030
> URL: https://issues.apache.org/jira/browse/IMPALA-9030
> Project: IMPALA
> Issue Type: Bug
> Reporter: Vihang Karajgaonkar
> Assignee: Vihang Karajgaonkar
> Priority: Major
>
> In HIVE-22158 HMS disallows creating of any managed table which is not
> transactional. This breaks the behavior for managed Kudu tables created from
> Impala. When user creates a managed Kudu table, HMS internally translates
> such table into a external table with an additional property
> "TRANSLATED_TO_EXTERNAL" set to true. The table type is changed to EXTERNAL.
> Subsequently, when such a table is dropped or renamed, Catalog thinks such
> tables as external and does not update Kudu (dropping the table in Kudu or
> renaming the table in Kudu). This is unexpected from the point of view of
> user since user may think that they created a managed table and Impala should
> handle the drop and rename accordingly. The same applies to certain other
> alter operations like alter table set properties on a managed Kudu table
> which previously was disallowed now goes through after HIVE-22158
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
