Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19878 )
Change subject: IMPALA-12131: Fix empty partition map in non-partitioned table when file metadata loading fails ...................................................................... IMPALA-12131: Fix empty partition map in non-partitioned table when file metadata loading fails When inserting non-partitioned tables, the catalog update request could fail due to file not found exceptions. At that point we have reset(cleared) the partition map so it becomes empty after the failure, which is an illegal state and will cause failures in later operations. Currently, users have to manually invalidate the metadata of the table to recover. We can improve this by making all the updates happen after all the external loadings succeed. So any failures in loading the file metadata won't leave the table metadata in a partially updated state. Testing: 1. Added a test which simulates a failure in a catalog update request by throwing an exception through the debug action and confirms that subsequent catalog update requests are not affected by the failure. Change-Id: I28e76a73b7905c24eb93b935124d20ea7abe8513 Reviewed-on: http://gerrit.cloudera.org:8080/19878 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/DebugUtils.java M tests/query_test/test_insert.py 4 files changed, 63 insertions(+), 7 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/19878 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I28e76a73b7905c24eb93b935124d20ea7abe8513 Gerrit-Change-Number: 19878 Gerrit-PatchSet: 34 Gerrit-Owner: ttttttz <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Manish Maheshwari <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: ttttttz <[email protected]>
