[
https://issues.apache.org/jira/browse/IMPALA-13293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Csaba Ringhofer updated IMPALA-13293:
-------------------------------------
Description:
Unlike ImpaladCatalog's waitForCatalogUpdate() the LocalCatalog version doesn't
use a conditional variable and simply waits for timeoutMs. The timeout comes
from MAX_CATALOG_UPDATE_WAIT_TIME_MS, which is 2 seconds. This means the the
function will wait 2 seconds even it the catalog update arrived in the
meantime. This 2 seconds is often nearly completely added to the impala cluster
startup time.
The sleep was added in
https://gerrit.cloudera.org/#/c/11472/3/fe/src/main/java/org/apache/impala/catalog/local/LocalCatalog.java
Update: realized that this also doesn't work well for ImpaladCatalog - the
FeCatalogManager creates a new ImpaladCatalog when a full topic update arrives,
so the Object that waitForCatalogUpdate() waits for is never notified. My
impression is that was broken a long time ago, even before LocalCatalog was
added.
was:
Unlike ImpaladCatalog's waitForCatalogUpdate() the LocalCatalog version doesn't
use a conditional variable and simply waits for timeoutMs. The timeout comes
from MAX_CATALOG_UPDATE_WAIT_TIME_MS, which is 2 seconds. This means the the
function will wait 2 seconds even it the catalog update arrived in the
meantime. This 2 seconds is often nearly completely added to the impala cluster
startup time.
The sleep was added in
https://gerrit.cloudera.org/#/c/11472/3/fe/src/main/java/org/apache/impala/catalog/local/LocalCatalog.java
> LocalCatalog's waitForCatalogUpdate() sleeps too much
> -----------------------------------------------------
>
> Key: IMPALA-13293
> URL: https://issues.apache.org/jira/browse/IMPALA-13293
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog, Frontend
> Reporter: Csaba Ringhofer
> Priority: Major
>
> Unlike ImpaladCatalog's waitForCatalogUpdate() the LocalCatalog version
> doesn't use a conditional variable and simply waits for timeoutMs. The
> timeout comes from MAX_CATALOG_UPDATE_WAIT_TIME_MS, which is 2 seconds. This
> means the the function will wait 2 seconds even it the catalog update arrived
> in the meantime. This 2 seconds is often nearly completely added to the
> impala cluster startup time.
> The sleep was added in
> https://gerrit.cloudera.org/#/c/11472/3/fe/src/main/java/org/apache/impala/catalog/local/LocalCatalog.java
> Update: realized that this also doesn't work well for ImpaladCatalog - the
> FeCatalogManager creates a new ImpaladCatalog when a full topic update
> arrives, so the Object that waitForCatalogUpdate() waits for is never
> notified. My impression is that was broken a long time ago, even before
> LocalCatalog was added.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
