Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/21072 )
Change subject: IMPALA-12831: Fix HdfsTable.toMinimalTCatalogObject() failed by concurrent modification ...................................................................... IMPALA-12831: Fix HdfsTable.toMinimalTCatalogObject() failed by concurrent modification HdfsTable.toMinimalTCatalogObject() is not always invoked with holding the table lock, e.g. in invalidating a table, we could replace an HdfsTable instance with an IncompleteTable instance. We then invoke HdfsTable.toMinimalTCatalogObject() to get the removed catalog object. However, the HdfsTable instance could be modified in the meantime by a concurrent DDL/DML that would reload it, e.g. a REFRESH statement. This causes HdfsTable.toMinimalTCatalogObject() failed by ConcurrentModificationException on the column/partition list. This patch fixes the issue by try acquiring the table read lock in HdfsTable.toMinimalTCatalogObject(). If it fails, the partition ids and names won't be returned. Also updates the method to not collecting the column list since it's unused. Tests - Added e2e test - Ran CORE tests Change-Id: Ie9f8e65c0bd24000241eedf8ca765c1e4e14fdb3 Reviewed-on: http://gerrit.cloudera.org:8080/21072 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/Table.java M tests/custom_cluster/test_concurrent_ddls.py 3 files changed, 70 insertions(+), 15 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/21072 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ie9f8e65c0bd24000241eedf8ca765c1e4e14fdb3 Gerrit-Change-Number: 21072 Gerrit-PatchSet: 5 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]>
