[
https://issues.apache.org/jira/browse/IMPALA-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar updated IMPALA-9664:
----------------------------------------
Summary: Support Hive replication for ACID tables (was: Support Hive
replication)
> Support Hive replication for ACID tables
> ----------------------------------------
>
> Key: IMPALA-9664
> URL: https://issues.apache.org/jira/browse/IMPALA-9664
> Project: IMPALA
> Issue Type: Bug
> Reporter: Vihang Karajgaonkar
> Assignee: Vihang Karajgaonkar
> Priority: Critical
> Fix For: Impala 4.0
>
>
> According to what we see in Hive source code, for transactional tables, the
> insert events are fired with a different API {{addWriteNotificationLog}}.
> Currently Impala fires {{firelistenerEvent}} for both transactional and
> non-transactional tables. We should look at what is the difference between
> the two APIs and see if we need to handle transactional tables differently.
> References:
> https://github.com/apache/hive/blob/c3afb57bdb1041f566fbbd896f625328fc9656a0/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2402
> https://github.com/apache/hive/blob/c3afb57bdb1041f566fbbd896f625328fc9656a0/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2236
> These insert events are used to replicate the changes in the ACID tables by
> tools like Hive replication. With the ability of insert data into ACID tables
> from Impala, we should also generate the insert events appropriately so that
> the replication works seemlessly. Additionally, the {{truncate table}}
> command should use the HMS API to truncate the table instead of deleteing the
> files directly from filesystem since it takes care of moving the files to a
> replication change management directory so that replication can have access
> to dropped data files.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
