Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14444 )
Change subject: IMPALA-8648: Add stress tests for ACID INSERTs/SELECTs ...................................................................... IMPALA-8648: Add stress tests for ACID INSERTs/SELECTs Added different stress tests for ACID operations. We already had one stress test where an Impala client issued SELECTs while another process inserted data through Hive. The test checked wether some invariants are held. I added the following tests: * one Impala client inserts data, one Impala client reads and checks invariants * multiple Impala clients issue INSERTs and TRUNCATEs, multiple Impala clients issue SELECTs and check invariants * multiple Impala clients insert data, sometimes failures are injected during inserts to create aborted transactions. In the meantime multiple Impala clients read the data and check that if they see data from aborted transactions During the tests I found and fixed a temporal deadlock. CatalogOpExecutor.truncateTable() locked catalog_.versionLock_ and the HMS ACID lock in a different order than the INSERT statements, it could lead to a deadlock for 30 seconds, after that TRUNCATE failed. Also, in the Frontend I switched the order of write id allocation and locking. From now on at first we lock the table then allocate a write id. Change-Id: I066652bfa7d924742af01aef8df4512e00620c7d Reviewed-on: http://gerrit.cloudera.org:8080/14444 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/Transaction.java M fe/src/main/java/org/apache/impala/common/TransactionKeepalive.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/Frontend.java M tests/common/impala_test_suite.py M tests/stress/test_acid_stress.py 7 files changed, 376 insertions(+), 104 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/14444 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I066652bfa7d924742af01aef8df4512e00620c7d Gerrit-Change-Number: 14444 Gerrit-PatchSet: 7 Gerrit-Owner: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Norbert Luksa <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
