Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11732 )
Change subject: IMPALA-7717: Handle concurrent partition changes in local catalog mode ...................................................................... IMPALA-7717: Handle concurrent partition changes in local catalog mode Current code throws a RuntimeException (RTE) when partial fetch RPCs looking up partition metadata and the corresponding partition ID is missing on the Catalog server. There are a couple of cases here. 1. The partition could be genuinely missing as it was dropped by a concurrent operation. 2. Partial fetch RPCs lookup partitions by IDs instead of names. This is problematic since the IDs can change over the lifetime of a table. In both the cases, throwing a RTE is not the right approach and for (2) we need to transparently retry the fetch with the new partition ID. We eventually need to fix (2) as looking up by partition ID is not the right approach. Testing: Updated an e-e test which fails without the patch. Change-Id: I2aa103ee159ce9478af9b5b27b36bc0cc286f442 Reviewed-on: http://gerrit.cloudera.org:8080/11732 Reviewed-by: Bharath Vissapragada <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M common/thrift/CatalogService.thrift M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java M fe/src/test/java/org/apache/impala/catalog/PartialCatalogInfoTest.java M tests/custom_cluster/test_local_catalog.py 5 files changed, 27 insertions(+), 58 deletions(-) Approvals: Bharath Vissapragada: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/11732 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I2aa103ee159ce9478af9b5b27b36bc0cc286f442 Gerrit-Change-Number: 11732 Gerrit-PatchSet: 8 Gerrit-Owner: Bharath Vissapragada <[email protected]> Gerrit-Reviewer: Adrian Ng (389) Gerrit-Reviewer: Bharath Vissapragada <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Vuk Ercegovac <[email protected]>
