Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16549 )
Change subject: IMPALA-6671: Skip locked tables from topic updates ...................................................................... Patch Set 7: (2 comments) http://gerrit.cloudera.org:8080/#/c/16549/7/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java: http://gerrit.cloudera.org:8080/#/c/16549/7/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@2678 PS7, Line 2678: public boolean updatePendingVersion(long expectedPendingVersion, I have a different idea below, but could we avoid the retry loop around updatePendingVersion() by considering it successful if catalogVersion_ > newPendingVersion. I think that would guarantee that the table gets picked up in the next topic update. http://gerrit.cloudera.org:8080/#/c/16549/7/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@2706 PS7, Line 2706: if (currentPendingVersion > version) { What if, instead of setCatalogVersion() checking pendingVersionNumber(), this was done lazily by the topic update thread. E.g. before calling getCatalogVersion(), the topic update thread does something like: tblVersion = getCatalogVersion(); // If setCatalogVersion() was called after the last time // the topic update thread checked the table. if (tblVersion > lastVersionSeenByTopicUpdate_) { // Bump catalogVersion_ if pending version is greater than previous update catalogVersion_.atomicMax(pendingVersionNumber_); } That would mean that only the topic thread needed to read/write pendingVersionNumber_, so there isn't the same race involving multiple versions. I don't know if there are any negative consequences from not updating https://stackoverflow.com/a/43210465/13270883 had some tips on doing an atomic max - looks like there are some clean solutions. -- To view, visit http://gerrit.cloudera.org:8080/16549 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic657b96edbcdc94c6b906e7ca59291f4e4715655 Gerrit-Change-Number: 16549 Gerrit-PatchSet: 7 Gerrit-Owner: Vihang Karajgaonkar <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Shant Hovsepian <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]> Gerrit-Comment-Date: Fri, 23 Oct 2020 22:54:28 +0000 Gerrit-HasComments: Yes
