[
https://issues.apache.org/jira/browse/IMPALA-14502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith resolved IMPALA-14502.
------------------------------------
Fix Version/s: Impala 5.0.0
Resolution: Fixed
> Redundant Metrics in IncompleteTable consuming extra space
> ----------------------------------------------------------
>
> Key: IMPALA-14502
> URL: https://issues.apache.org/jira/browse/IMPALA-14502
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Critical
> Fix For: Impala 5.0.0
>
> Attachments: Dominator.png, Histogram.png
>
>
> In a catalogd heap dump where all of the tables are unloaded, we found
> IncompleteTable consumes more memory space than just the strings of db/table
> name and table type/comment.
> !Histogram.png|width=676,height=436!
> As shown in the histogram, there are 2.6M instances of IncompleteTable
> consuming around 18GB of the heap space. Each instance takes around 7KB of
> memory.
> Looking into the dominator tree (group by classes) of IncompleteTable
> instances, the majority of the space is consumed by Metrics which will never
> be used for IncompleteTable (see
> [code|https://github.com/apache/impala/blob/ebbc67cf40bd856253d07c649028888d85c772cc/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java#L4242-L4244]).
> !Dominator.png|width=732,height=618!
> We should ignore initializing these metrics for IncompleteTable.
> {code:java}
> public void initMetrics() {
> metrics_.addTimer(REFRESH_DURATION_METRIC);
> metrics_.addTimer(ALTER_DURATION_METRIC);
> metrics_.addTimer(LOAD_DURATION_METRIC);
> metrics_.addTimer(LOAD_DURATION_STORAGE_METADATA);
> metrics_.addTimer(HMS_LOAD_TBL_SCHEMA);
> metrics_.addTimer(LOAD_DURATION_ALL_COLUMN_STATS);
> metrics_.addCounter(NUMBER_OF_INFLIGHT_EVENTS);
> metrics_.addTimer(TBL_EVENTS_PROCESS_DURATION);
> metrics_.addGauge(LAST_SYNC_EVENT_ID,
> (Gauge<Long>) () -> Long.valueOf(lastSyncedEventId_));
> }{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
