Tim Armstrong has posted comments on this change. Change subject: IMPALA-4733: Avoid using several loading threads on one table. ......................................................................
Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/5707/1/fe/src/main/java/org/apache/impala/catalog/TableLoadingMgr.java File fe/src/main/java/org/apache/impala/catalog/TableLoadingMgr.java: Line 280: tableLoadingSet_.remove(tblName); There's still a window for a race here if multiple threads pull the same entry off the queue at the same time. I think we can avoid it if we change the data structures slightly. How about we change loadingTables_ to a map from TTableName to an atomic integer, and only only remove tables from the map once the load finishes? That way, the first loading thread to flip the atomic integer from 0 to 1 wins the race and does the load. If another thread doesn't find the entry it knows that a load completed after it was last added. If it finds the integer set to 1, it knows that a load is in progress. Not ideal but it makes the race impossible and keeps this mechanism separate from loadingTables_, which has its own lifecycle. -- To view, visit http://gerrit.cloudera.org:8080/5707 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Idba5f1808e0b9cbbcf46245834d8ad38d01231cb Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-HasComments: Yes
