bonampak opened a new pull request, #1392: URL: https://github.com/apache/knox/pull/1392
[KNOX-3444](https://issues.apache.org/jira/browse/KNOX-3444) - Exclude role types that are not referenced by ServiceModelGenerators ## What changes were proposed in this pull request? During CM discovery only cache role configurations that ServiceModelGenerators actually use. ## How was this patch tested? Updated unit tests and did manual test on a cluster. Verified that the ClusterConfiguration cache persisted after discovery is the same as before the changes were applied and the topologies have the same generated service URLs. Acquired a heap dump using jmap: ```bash jmap -dump:live,format=b,file="/tmp/knox-heap-jmap-${knox_pid}".hprof "${knox_pid}"; ``` Used Eclipse Memory Analyzer Tool and Calcite SQL plugin to analyze the heap usage of the ApiRoleConfig objects cached in `ClouderaManagerServiceDiscoveryRepository` ```sql SELECT toString(r.this['roleType']) roleType, COUNT(r.this) as cnt, AVG(retainedSize(r.this)) as avg_retained_size, SUM(retainedSize(r.this)) sum_retained_size FROM "com.cloudera.api.swagger.model.ApiRoleConfig" r GROUP BY toString(r.this['roleType']) ORDER BY sum_retained_size desc, roleType ``` The unused roles (e.g. NODEMANAGER, REGIONSERVER, GATEWAY, DATANODE, KNOX_GATEWAY, KAFKA_BROKER, OZONE_DATANODE, etc.) are not cached. For each unused role, the memory savings are 60K - 150K by role instance not cached (with an average of roughly 100K). ## Integration Tests N/A. Integration tests would require a live CM dependency. Setting up a real (or faithfully simulated) Cloudera Manager server with representative services/roles is heavy infrastructure that the project's test suite does not provide, and a mock-based "integration" test would exercise the same code paths the unit tests already cover. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
