Hello Yongzhi Chen, Gabor Kaszab, Tim Armstrong, Csaba Ringhofer, Impala Public
Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/13968
to look at the new patch set (#7).
Change subject: IMPALA-8637: Implement transaction handling and locking for
ACID queries
......................................................................
IMPALA-8637: Implement transaction handling and locking for ACID queries
Adds background thread to the Frontend that periodically heartbeats
the opened transactions and locks. Most of the logic is implemented
in the newly added TransactionKeepalive class.
TransactionKeepalive keeps track of the creation time of the
transactions and locks, and only heartbeat them if they are older
then the sleep interval. This way we don't heartbeat short-running
queries unnecessarily.
Testing:
Added an exhaustive test that checks heartbeating. It creates a long
running transaction via a debug action injects sleeps into the
HdfsTableSink. While the transaction is executing the test periodically
checks whether there is an OPEN transaction that has sent a heartbeat
to HMS. If it found one then the test succeeds.
TODOs:
* add metrics
Change-Id: Iaa37899b24aa114be642bf8772b4e0f882865cfa
---
M be/src/exec/hdfs-table-sink.cc
M be/src/service/client-request-state.cc
M be/src/service/frontend.cc
M be/src/service/frontend.h
M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
A fe/src/main/java/org/apache/impala/service/TransactionKeepalive.java
M tests/query_test/test_acid.py
10 files changed, 516 insertions(+), 20 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/13968/7
--
To view, visit http://gerrit.cloudera.org:8080/13968
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iaa37899b24aa114be642bf8772b4e0f882865cfa
Gerrit-Change-Number: 13968
Gerrit-PatchSet: 7
Gerrit-Owner: Zoltan Borok-Nagy <borokna...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <gaborkas...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
Gerrit-Reviewer: Yongzhi Chen <yc...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com>
