Hello Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16602
to look at the new patch set (#2).
Change subject: IMPALA-10243: ConcurrentModificationException during parallel
INSERTs
......................................................................
IMPALA-10243: ConcurrentModificationException during parallel INSERTs
Impala might throw a ConcurrentModificationException during a high
load of INSERTs to the same table. The exception happens during thrift
serialization of TUpdateCatalogResponse which have a reference to the
metastore table. The serialization happens without a lock, so another
thread might modify the metastore table object in the meantime.
This can potentially happen in CatalogOpExecutor.updateCatalog() which
updates the catalog version and unsets table column statistics.
For some reason I only saw this error with local catalog.
The problem is that in Table.toThrift() we set a reference to the
metastore table object instead of deep copying it. So my fix is to deep
copy the metastore table, this prevents concurrent modifications.
Testing
* added stress test 'test_insert_stress.py'
Change-Id: Ie656925d764d5eb26c318703ca425529ecf7a3a3
---
M fe/src/main/java/org/apache/impala/catalog/Table.java
A tests/stress/stress_util.py
M tests/stress/test_acid_stress.py
A tests/stress/test_insert_stress.py
4 files changed, 150 insertions(+), 23 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/02/16602/2
--
To view, visit http://gerrit.cloudera.org:8080/16602
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie656925d764d5eb26c318703ca425529ecf7a3a3
Gerrit-Change-Number: 16602
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
