Alex Behm has uploaded a new change for review. http://gerrit.cloudera.org:8080/5707
Change subject: IMPALA-4733: Avoid using several loading threads on one table. ...................................................................... IMPALA-4733: Avoid using several loading threads on one table. When there are multiple concurrent requests to the catalogd to prioritize loading the same table, then several catalog loading threads may end up waiting for that single table to be loaded, effectively reducing the number of catalog loading threads. In extreme examples, this might degrade to serial loading of tables. The existing design and data structures in the catalog service make this bug somewhat tricky to fix completely, so this patch adds a simpe fix that addresses the most common and severe manifestations of this bug. It prevents wasting several threads on long-running table loads by checking the map of in-progress table loads before waiting on the future of that table. Testing: I could easily reproduce the bug locally with the steps described in the JIRA. After this patch, I could not observe threads being wasted anymore. Change-Id: Idba5f1808e0b9cbbcf46245834d8ad38d01231cb --- M fe/src/main/java/org/apache/impala/catalog/TableLoadingMgr.java 1 file changed, 7 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/07/5707/1 -- To view, visit http://gerrit.cloudera.org:8080/5707 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Idba5f1808e0b9cbbcf46245834d8ad38d01231cb Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm <[email protected]>
