Impala Public Jenkins has submitted this change and it was merged. Change subject: IMPALA-4765: Avoid using several loading threads on one table. ......................................................................
IMPALA-4765: Avoid using several loading threads on one table. When there are multiple concurrent requests to the catalogd to prioritize loading the same table, then several catalog loading threads may end up waiting for that single table to be loaded, effectively reducing the number of catalog loading threads. In extreme examples, this might degrade to serial loading of tables. This patch augments the existing data structures and code to prevent using several loading threads for the same table. Some of the existing data structures and code could be consolidated/simplified but this patch does not try to address that issue to minimize the risk of this change. Testing: I could easily reproduce the bug locally with the steps described in the JIRA. After this patch, I could not observe threads being wasted anymore. Change-Id: Idba5f1808e0b9cbbcf46245834d8ad38d01231cb Reviewed-on: http://gerrit.cloudera.org:8080/5707 Reviewed-by: Alex Behm <[email protected]> Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/catalog/TableLoadingMgr.java 1 file changed, 34 insertions(+), 21 deletions(-) Approvals: Impala Public Jenkins: Verified Alex Behm: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5707 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Idba5f1808e0b9cbbcf46245834d8ad38d01231cb Gerrit-PatchSet: 7 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm <[email protected]> Gerrit-Reviewer: Alex Behm <[email protected]> Gerrit-Reviewer: Bharath Vissapragada <[email protected]> Gerrit-Reviewer: Dimitris Tsirogiannis <[email protected]> Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]>
