Alex Behm has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/5707

Change subject: IMPALA-4733: Avoid using several loading threads on one table.
......................................................................

IMPALA-4733: Avoid using several loading threads on one table.

When there are multiple concurrent requests to the catalogd to
prioritize loading the same table, then several catalog loading
threads may end up waiting for that single table to be loaded,
effectively reducing the number of catalog loading threads. In
extreme examples, this might degrade to serial loading of tables.

The existing design and data structures in the catalog service
make this bug somewhat tricky to fix completely, so this patch adds
a simpe fix that addresses the most common and severe manifestations
of this bug. It prevents wasting several threads on long-running table
loads by checking the map of in-progress table loads before waiting on
the future of that table.

Testing: I could easily reproduce the bug locally with the steps
described in the JIRA. After this patch, I could not observe threads
being wasted anymore.

Change-Id: Idba5f1808e0b9cbbcf46245834d8ad38d01231cb
---
M fe/src/main/java/org/apache/impala/catalog/TableLoadingMgr.java
1 file changed, 7 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/07/5707/1
-- 
To view, visit http://gerrit.cloudera.org:8080/5707
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Idba5f1808e0b9cbbcf46245834d8ad38d01231cb
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Alex Behm <alex.b...@cloudera.com>

Reply via email to