Hello Yongzhi Chen, Vihang Karajgaonkar, Tim Armstrong, Csaba Ringhofer, Dinesh
Garg, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/14071
to look at the new patch set (#7).
Change subject: IMPALA-8793: Implement TRUNCATE for insert-only ACID tables
......................................................................
IMPALA-8793: Implement TRUNCATE for insert-only ACID tables
This commit adds support for the TRUNCATE statement for transactional
tables. TRUNCATE on transactional tables doesn't remove the files, it
just creates new empty ACID base directories.
Although from a filessystem point of view the directories won't be
completely empty. They will contain the hidden '_empty' file, so
FileSystemUtil.listFiles() will list them. It's similar to Hive's
behavior which creates the magic file '_orc_acid_version' in each
transactional directory. Except Impala only creates these '_empty'
files in empty base directories.
All the functionality is implemented in the catalog, therefore
transaction handling, table locking, heartbeating also happens there.
Added the new Transaction class that should prevent leakage of
transactions, it should be used in try-with-resources statements.
Testing:
Added backend tests that truncate non-partitioned and partitioned
tables as well.
The tests check whether the statistics were removed.
The tests also check if Hive sees the effect of truncation.
Change-Id: Ic749b7f27da157e1c0ebf9b7e9b6ee09afad122a
---
M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/main/java/org/apache/impala/analysis/TruncateStmt.java
M fe/src/main/java/org/apache/impala/catalog/Catalog.java
A fe/src/main/java/org/apache/impala/catalog/Transaction.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzerTest.java
A testdata/workloads/functional-query/queries/QueryTest/acid-truncate.test
M tests/metadata/test_hms_integration.py
M tests/query_test/test_acid.py
9 files changed, 353 insertions(+), 23 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/71/14071/7
--
To view, visit http://gerrit.cloudera.org:8080/14071
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic749b7f27da157e1c0ebf9b7e9b6ee09afad122a
Gerrit-Change-Number: 14071
Gerrit-PatchSet: 7
Gerrit-Owner: Zoltan Borok-Nagy <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Dinesh Garg (430)
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Tim Armstrong <[email protected]>
Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]>
Gerrit-Reviewer: Yongzhi Chen <[email protected]>
Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
