Zoltán Borók-Nagy created IMPALA-10243:
------------------------------------------
Summary: ConcurrentModificationException during parallel INSERTs
Key: IMPALA-10243
URL: https://issues.apache.org/jira/browse/IMPALA-10243
Project: IMPALA
Issue Type: Bug
Reporter: Zoltán Borók-Nagy
Assignee: Zoltán Borók-Nagy
Impala might throw a ConcurrentModificationException during a high load of
INSERTs to the same table.
The exception happens during thrift serialization of TUpdateCatalogResponse
which have a reference to the metastore table. The serialization happens
without a lock, so another thread might modify the metastore table object in
the meantime. This can potentially happen in CatalogOpExecutor.updateCatalog()
which updates the catalog version and unsets table column statistics. A high
load of INSERT statements increase the probability of the concurrent
modification.
I think the problem is that in Table.toThrift() we set a reference to the
metastore table object instead of deep copying it:
https://github.com/apache/impala/blob/481ea4ab0d476a4aa491f99c2a4e376faddc0b03/fe/src/main/java/org/apache/impala/catalog/Table.java#L505
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
