Repository: impala Updated Branches: refs/heads/master 083352b1e -> 15e40a3c9
IMPALA-7498: Fix log spew from LocalCatalog startup Frontend calls LocalCatalog#waitForCatalog() in a tight loop during startup and ends up spewing tons of log messages if the MetaProvider takes some time to initialize. This is a problem for CatalogdMetaProvider implementation since it waits on the Catalog server to send an initial update. This can take some time depending on various factors and the logs become too noisy during that period. This patch adds a sleep() inside waitForCatalog() to avoid this. We can do something fancy, like ImpaladCatalog implementation, by waiting on a synchronized monitor and getting notified when the Catalog is initialized, but I don't think it is worth the effort, especially since the sleep time is pretty small (MAX_CATALOG_UPDATE_WAIT_TIME_MS = 2s). Testing: Tested it locally by just starting the impalad catalog in local mode and the logs look much better while waiting for the Catalog to send the initial update. Change-Id: I81e7cc6f464fd71afecd94a5a2b777aff47ee11b Reviewed-on: http://gerrit.cloudera.org:8080/11472 Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Reviewed-by: Vuk Ercegovac <vercego...@cloudera.com> Project: http://git-wip-us.apache.org/repos/asf/impala/repo Commit: http://git-wip-us.apache.org/repos/asf/impala/commit/d81d5086 Tree: http://git-wip-us.apache.org/repos/asf/impala/tree/d81d5086 Diff: http://git-wip-us.apache.org/repos/asf/impala/diff/d81d5086 Branch: refs/heads/master Commit: d81d5086bf29bd81a30fe1ab5287615ee613c02f Parents: 083352b Author: Bharath Vissapragada <bhara...@cloudera.com> Authored: Wed Sep 19 11:50:44 2018 -0700 Committer: Bharath Vissapragada <bhara...@cloudera.com> Committed: Mon Sep 24 17:31:05 2018 +0000 ---------------------------------------------------------------------- .../org/apache/impala/catalog/local/LocalCatalog.java | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/impala/blob/d81d5086/fe/src/main/java/org/apache/impala/catalog/local/LocalCatalog.java ---------------------------------------------------------------------- diff --git a/fe/src/main/java/org/apache/impala/catalog/local/LocalCatalog.java b/fe/src/main/java/org/apache/impala/catalog/local/LocalCatalog.java index 328f3b1..683ed18 100644 --- a/fe/src/main/java/org/apache/impala/catalog/local/LocalCatalog.java +++ b/fe/src/main/java/org/apache/impala/catalog/local/LocalCatalog.java @@ -208,10 +208,13 @@ public class LocalCatalog implements FeCatalog { @Override public void waitForCatalogUpdate(long timeoutMs) { - // No-op for local catalog. - // TODO(todd): Frontend.waitForCatalog() gets called at startup and ends - // up being a tight loop that spews logs unless we do something to wait better - // here. + if (isReady()) return; + // Sleep here to avoid log spew from the retry loop in Frontend. + try { + Thread.sleep(timeoutMs); + } catch (InterruptedException e) { + // Ignore + } } @Override
