[jira] [Commented] (IMPALA-7498) impalad should wait for catalogd during start up
[ https://issues.apache.org/jira/browse/IMPALA-7498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16626450#comment-16626450 ] ASF subversion and git services commented on IMPALA-7498: - Commit d81d5086bf29bd81a30fe1ab5287615ee613c02f in impala's branch refs/heads/master from Bharath Vissapragada [ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=d81d508 ] IMPALA-7498: Fix log spew from LocalCatalog startup Frontend calls LocalCatalog#waitForCatalog() in a tight loop during startup and ends up spewing tons of log messages if the MetaProvider takes some time to initialize. This is a problem for CatalogdMetaProvider implementation since it waits on the Catalog server to send an initial update. This can take some time depending on various factors and the logs become too noisy during that period. This patch adds a sleep() inside waitForCatalog() to avoid this. We can do something fancy, like ImpaladCatalog implementation, by waiting on a synchronized monitor and getting notified when the Catalog is initialized, but I don't think it is worth the effort, especially since the sleep time is pretty small (MAX_CATALOG_UPDATE_WAIT_TIME_MS = 2s). Testing: Tested it locally by just starting the impalad catalog in local mode and the logs look much better while waiting for the Catalog to send the initial update. Change-Id: I81e7cc6f464fd71afecd94a5a2b777aff47ee11b Reviewed-on: http://gerrit.cloudera.org:8080/11472 Tested-by: Impala Public Jenkins Reviewed-by: Vuk Ercegovac > impalad should wait for catalogd during start up > > > Key: IMPALA-7498 > URL: https://issues.apache.org/jira/browse/IMPALA-7498 > Project: IMPALA > Issue Type: Sub-task >Reporter: Todd Lipcon >Assignee: bharath v >Priority: Major > Fix For: Impala 3.1.0 > > > If you start all daemons simultaneously, impalad with --use_local_catalog > enabled will retry three times in a tight loop trying to fetch the DB names, > and then exit. Instead it should loop for some amount of time waiting for the > catalog to be ready in the same way that the existing implementation does. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7498) impalad should wait for catalogd during start up
[ https://issues.apache.org/jira/browse/IMPALA-7498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621102#comment-16621102 ] bharath v commented on IMPALA-7498: --- Todd, I looked at the code and I don't think it will retry 3 times and then exit. It may be true in the earlier versions of V2 feature but I'm not able to track down the 3-time retry loop. Please correct me if I'm wrong. It waits indefinitely for the initial update but the issue happens to be with the log-spew. I've sent out of a patch here http://gerrit.cloudera.org:8080/11472 to fix it. > impalad should wait for catalogd during start up > > > Key: IMPALA-7498 > URL: https://issues.apache.org/jira/browse/IMPALA-7498 > Project: IMPALA > Issue Type: Sub-task >Reporter: Todd Lipcon >Assignee: bharath v >Priority: Major > > If you start all daemons simultaneously, impalad with --use_local_catalog > enabled will retry three times in a tight loop trying to fetch the DB names, > and then exit. Instead it should loop for some amount of time waiting for the > catalog to be ready in the same way that the existing implementation does. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7498) impalad should wait for catalogd during start up
[ https://issues.apache.org/jira/browse/IMPALA-7498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596754#comment-16596754 ] Philip Zeyliger commented on IMPALA-7498: - There's a similar issue with the catalog, which fails (in an unpleasant, core-dump-y way) if it can't talk to the Hive metastore. To be consistent, we should wait a certain amount of time in all cases. (No obligation to fix this together; just mentioning it if you end up in the same code.) > impalad should wait for catalogd during start up > > > Key: IMPALA-7498 > URL: https://issues.apache.org/jira/browse/IMPALA-7498 > Project: IMPALA > Issue Type: Sub-task >Reporter: Todd Lipcon >Priority: Major > > If you start all daemons simultaneously, impalad with --use_local_catalog > enabled will retry three times in a tight loop trying to fetch the DB names, > and then exit. Instead it should loop for some amount of time waiting for the > catalog to be ready in the same way that the existing implementation does. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
