Yana, The full integration test passed without issues. Can you create a PR of your findings and the change so we can document it and give credit?
-Doug On Friday, December 13, 2024 at 1:44:47 AM UTC-5 Doug Hoard wrote: > Using String intern() can help in some scenarios and cause performance > issues in other scenarios. > > I made your change on my development branch and ran a quick test > (./quick-test.sh) and didn't see any unit test or integration test issues. > > I just started a full regression test and will have some results in the > morning (US time.) (It takes a little over 3 hours.) > > -Doug > > On Thursday, December 12, 2024 at 11:02:02 AM UTC-5 Яна Илиева wrote: > >> Hello, >> >> While using the Prometheus JMX Exporter as a Java agent in our >> application, we observed frequent GC clean ups and it turns out it is >> because the available heap space got exhausted by the caching of the >> Prometheus JMX Exporter. >> >> For context - we have an exporter yaml file with about 40 rules defined >> and there are many JMX MBeans which we process in order to get the required >> metrics. Without caching enabled for the rules, the request can take a >> minimum of 1 minute to complete. In an attempt to reduce this time, we >> enabled caching and observed that the MatchedRulesCache object can take >> around 600 MB in a particular case. There is a realistic potential in out >> case that this object grows above 1 GB, which is a huge amount of space for >> this kind of process. >> >> We identified that the major reason for the large size of >> MatchedRulesCache is that it contains duplicating String objects for the >> same MBean names. The object keeps all MBean names the rules were matched >> against previously, in order to avoid expensive pattern matching later. >> Although each rule caches the same set of MBean names, those are in fact >> separate objects in the heap. >> >> The origin of the matter was found in >> https://github.com/prometheus/jmx_exporter/blob/a3dac9acee1464531cd87502579178a1fec1cc76/collector/src/main/java/io/prometheus/jmx/JmxCollector.java#L584 >> >> where for each rule with caching enabled, there is a new String object >> created, although such could already exist in memory from a previous >> iteration. >> >> The duplication is by a factor of the number of rules with caching >> enabled. >> >> I experimented with interning the String, >> >> *String matchName = (beanName + attributeName + ": " + >> matchBeanValue).intern();* >> >> so that the JVM's string pool is utilized and the String objects reused. >> The result in speed and heap space used by cache is described below: >> >> [image: Screenshot from 2024-12-12 14-53-26.png] >> >> What do you think about this and do you find this suggestion could have >> negative consequences in certain cases? >> >> Kind regards, >> Yana Ilieva >> > -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsubscr...@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/prometheus-developers/da7d418b-5fd4-4946-84dd-4d2897053190n%40googlegroups.com.
