Andrew Wong created IMPALA-10557:
------------------------------------
Summary: Allow Impala to use Kudu's multi-row transaction
implementation
Key: IMPALA-10557
URL: https://issues.apache.org/jira/browse/IMPALA-10557
Project: IMPALA
Issue Type: New Feature
Components: Clients
Reporter: Andrew Wong
Once Kudu's support for multi-row transactions materializes, it'd be great if
Impala could expose a query option or session-level flag to indicate that
Impala should run queries that insert into Kudu in the context of a single
transaction.
When the query option is set, similar to how transactions are opened for Hive
ACID tables, a new Kudu transaction should be started by Impala's Frontend, and
the resulting KuduTransaction object should be serialized and passed as context
to the Impala executors, which should open their Kudu sessions using the
{{KuduTransaction::CreateSession()}} API instead of the usual
{{KuduClient::NewSession()}}.
Errors returned in the session should be propagated to the Impala Frontend
before committing – it seems reasonable to expect that any session errors
should result in Impala aborting the transaction.
The {{KuduTransaction}} handle in the frontend has a heartbeater built in, so
unlike Hive ACID transactions, which require manual tracking of heartbeating,
no additional manual requests are needed to maintain the transaction.
Kudu's Java client API is defined in [this
patch|https://gerrit.cloudera.org/c/16894/], and C++ client API is defined in
[this patch|https://gerrit.cloudera.org/c/16710/]. The implementation isn't
fully operational quite yet – once [this patch to automatically register
participants|https://gerrit.cloudera.org/c/17037/] lands, Impala developers
should at least be able to start adding basic testing non-overlapping cases
(partition locking is still in progress).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
