Tim Armstrong has posted comments on this change.

Change subject: IMPALA-4733: Avoid using several loading threads on one table.
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/5707/1/fe/src/main/java/org/apache/impala/catalog/TableLoadingMgr.java
File fe/src/main/java/org/apache/impala/catalog/TableLoadingMgr.java:

Line 280:     tableLoadingSet_.remove(tblName);
There's still a window for a race here if multiple threads pull the same entry 
off the queue at the same time. I think we can avoid it if we change the data 
structures slightly.

How about we change loadingTables_ to a map from TTableName to an atomic 
integer, and only only remove tables from the map once the load finishes?

That way, the first loading thread to flip the atomic integer from 0 to 1 wins 
the race and does the load. If another thread doesn't find the entry it knows 
that a load completed after it was last added. If it finds the integer set to 
1, it knows that a load is in progress.

Not ideal but it makes the race impossible and keeps this mechanism separate 
from loadingTables_, which has its own lifecycle.


-- 
To view, visit http://gerrit.cloudera.org:8080/5707
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Idba5f1808e0b9cbbcf46245834d8ad38d01231cb
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Alex Behm <alex.b...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
Gerrit-HasComments: Yes

Reply via email to