Zoltan Borok-Nagy has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16602
Change subject: IMPALA-10243: ConcurrentModificationException during parallel INSERTs ...................................................................... IMPALA-10243: ConcurrentModificationException during parallel INSERTs Impala might throw a ConcurrentModificationException during a high load of INSERTs to the same table. The exception happens during thrift serialization of TUpdateCatalogResponse which have a reference to the metastore table. The serialization happens without a lock, so another thread might modify the metastore table object in the meantime. This can potentially happen in CatalogOpExecutor.updateCatalog() which updates the catalog version and unsets table column statistics. For some reason I only saw this error with local catalog. The problem is that in Table.toThrift() we set a reference to the metastore table object instead of deep copying it. So my fix is to deep copy the metastore table, this prevents concurrent modifications. Testing * added stress test 'test_insert_stress.py' Change-Id: Ie656925d764d5eb26c318703ca425529ecf7a3a3 --- M fe/src/main/java/org/apache/impala/catalog/Table.java A tests/stress/stress_util.py M tests/stress/test_acid_stress.py A tests/stress/test_insert_stress.py 4 files changed, 149 insertions(+), 23 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/02/16602/1 -- To view, visit http://gerrit.cloudera.org:8080/16602 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ie656925d764d5eb26c318703ca425529ecf7a3a3 Gerrit-Change-Number: 16602 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy <[email protected]>
