[
https://issues.apache.org/jira/browse/KNOX-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18114204#comment-18114204
]
ASF subversion and git services commented on KNOX-3444:
-------------------------------------------------------
Commit 8697bc73677e46f8afbc4b442cf39a8aa5188576 in knox's branch
refs/heads/master from bonampak
[ https://gitbox.apache.org/repos/asf?p=knox.git;h=8697bc736 ]
KNOX-3444: Optimize CM discovery memory usage by only caching role
configurations that ServiceModelGenerators actually use (#1392)
* KNOX-3444: Skip caching CM discovery role configs for role types no
ServiceModelGenerator uses
* KNOX-3444: Filter each page as it arrives in ServiceRoleCollectorByService so
unneeded configs do not accumulate.
* KNOX-3444: When requiredTypeNames is empty in TypeNameFilter, exclude every
role as no service model would be generated.
> Optimize CM discovery memory usage by only caching role configurations that
> ServiceModelGenerators actually use
> ---------------------------------------------------------------------------------------------------------------
>
> Key: KNOX-3444
> URL: https://issues.apache.org/jira/browse/KNOX-3444
> Project: Apache Knox
> Issue Type: Task
> Components: cm-discovery
> Reporter: Tamás Marcinkovics
> Assignee: Tamás Marcinkovics
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Currently CM auto discovery fetches role configurations for every service
> role that is not explicitly excluded, using full view.
> The response role config items include a lot of metadata, such as description
> and validation, for every role instance in the service.
> For some role types that currently no ServiceModelGenerators use (DATANODE,
> GATEWAY, NODEMANAGER, REGIONSERVER) the role config items list in the
> response takes up around 200KB memory; for large clusters with 500+ nodes,
> this will result in a heap usage of 40-100 MB per role type, and the
> in-memory discovery cache can exceed 1GB.
> We should not cache the role config items not used by ServiceModelGenerators.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
