Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19878 )
Change subject: IMPALA-12131: For non-partitioned tables, defer the resetPartitions() until the file metadata is loaded. ...................................................................... Patch Set 19: Code-Review+1 (4 comments) http://gerrit.cloudera.org:8080/#/c/19878/19//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19878/19//COMMIT_MSG@9 PS19, Line 9: For non-partitioned tables, if the catalog update request : fails after resetting the partition, its partition map size : will be 0, which will cause the next catalog update requests : to fail due to Exception:"java.util.NoSuchElementException". We'd better refine this long sentence, e.g. "When inserting non-partitioned tables, the catalog update request could fail due to file not found exceptions. At that point we have reset(cleared) the partition map so it becomes empty after the failure, which is an illegal state and will cause failures in later operations." http://gerrit.cloudera.org:8080/#/c/19878/19//COMMIT_MSG@13 PS19, Line 13: Currently, the only solution to this problem is to invalidate : the metadata of the table. "Currently, users have to manually invalidate the metadata of the table to recover" http://gerrit.cloudera.org:8080/#/c/19878/19//COMMIT_MSG@14 PS19, Line 14: For non-partitioned tables, defer : the resetPartitions() until the file metadata is loaded. "We can improve this by making all the updates happen after all the external loadings succeed. So any failures in loading the file metadata won't leave the table metadata in a partially updated state." http://gerrit.cloudera.org:8080/#/c/19878/19/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java: http://gerrit.cloudera.org:8080/#/c/19878/19/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@1397 PS19, Line 1397: setUnpartitionedTableStats(partBuilder); nit: setUnpartitionedTableStats() also updates internal fields. Please move it after the comment of "Step 2". -- To view, visit http://gerrit.cloudera.org:8080/19878 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I28e76a73b7905c24eb93b935124d20ea7abe8513 Gerrit-Change-Number: 19878 Gerrit-PatchSet: 19 Gerrit-Owner: ttttttz <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Manish Maheshwari <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: ttttttz <[email protected]> Gerrit-Comment-Date: Tue, 27 Jun 2023 02:42:38 +0000 Gerrit-HasComments: Yes
