[
https://issues.apache.org/jira/browse/IMPALA-5835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
bharath v resolved IMPALA-5835.
-------------------------------
Resolution: Cannot Reproduce
Fix Version/s: Not Applicable
Please try out the latest bits and reopen if needed.
> Severe slowdown in catalogd startup after 2.1 → 2.5 upgrade with > 200,000
> databases
> ------------------------------------------------------------------------------------
>
> Key: IMPALA-5835
> URL: https://issues.apache.org/jira/browse/IMPALA-5835
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 2.6.0, Impala 2.7.0, Impala 2.5.5, Impala 2.8.0,
> Impala 2.9.0, Impala 2.10.0
> Reporter: Ben Breakstone
> Assignee: bharath v
> Priority: Major
> Labels: performance
> Fix For: Not Applicable
>
>
> After an upgrade from Impala 2.1 (CDH 5.3.9) to Impala 2.5 (CDH 5.7.5),
> starting up Catalog Server takes around eight to ten hours. It took around
> twenty minutes before the upgrade.
> There are over 200,000 databases in use. Looking in the catalogd log as it
> starts up for hours, it says
> "Loading native functions for database..." and then
> "Loading Java functions for database..." for each database. Based on this, it
> appears the introduction of persistent UDFs/UDAs is causing the slowdown.
> Only one of the databases actually has any UDFs defined.
> num_metadata_loading_threads is set to 64. Background loading of metadata is
> disabled.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
