[
https://issues.apache.org/jira/browse/IMPALA-14739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18058092#comment-18058092
]
ASF subversion and git services commented on IMPALA-14739:
----------------------------------------------------------
Commit ffcb9a9ec74b9797e2e1b231f3aa1ca074da35a4 in impala's branch
refs/heads/master from Csaba Ringhofer
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=ffcb9a9ec ]
IMPALA-14739: Log if local catalog cache entry size is truncated
The weigh() function cannot return more than 2GB. Such huge objects
should be very rare, but if it happens, the cache will underestimate
its size. Add a warning in this case.
Change-Id: I30554e55e0d487be38e10ddadb7fe53410f61201
Reviewed-on: http://gerrit.cloudera.org:8080/23964
Reviewed-by: Csaba Ringhofer <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Local catalog cache mem usage is hard do diagnise
> -------------------------------------------------
>
> Key: IMPALA-14739
> URL: https://issues.apache.org/jira/browse/IMPALA-14739
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog, Frontend
> Reporter: Csaba Ringhofer
> Priority: Critical
>
> Huge objects like Iceberg tables with many files can dominate cache memory
> usage, but it is hard to decipher this information from metrics. There is
> also no trace of the very problematic case of truncating weight of >2GB
> objects:
> https://github.com/apache/impala/blob/3be15fd3598071eaeddd9b4d29e0883b95fdd14a/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java#L2353
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
