On Friday, 5 June 2020 22:31:13 UTC+12, Brian Brazil wrote: > > On Fri, 5 Jun 2020 at 11:13, Cameron Kerr <[email protected] > <javascript:>> wrote: > >> Hi all, I've spent a few days coming up to speed on understanding >> jmx_exporter, and I think I'm in a pretty good place now, understanding how >> MBeans, JMX, and RMI work. >> >> I've so far deployed jmx_exporter in two ways: >> >> * as a Java Agent on Tomcat 6 (on RHEL6) for a small application that >> includes SOLR and PostgreSQL >> * and as a standalone Java HTTP server on Tomcat 8.5.34 (that comes >> bundled with a mission-critical application) >> >> I found going with the Java Agent relatively easy, although I think I'll >> contribute a blog post and pull-request to help on the documentation front. >> >> You might reasonably ask why I'm bothering with the HTTP server. Here's >> my business logic that drives this: >> >> * we have a mission-critical application that we urgently need to improve >> our visibility on to diagnose some performance limits we believe we're >> reaching >> * we're reluctant to introduce (and cause an outage to introduce) a java >> agent --- as far as I'm aware jmx_exporter lacks the ability that jconsole >> has to dynamically inject an agent. >> > > It's a very simple Java agent that doesn't do anything fancy. I believe > it's possible if you already know how to do such things, though using it as > an agent is almost always better. >
I can appreciate that a lot more now that I understand more about the issues around RMI etc. > * as part of a previous monitoring drive, we've already introduced the >> appropriate Remote JMX configuration (-D....jmxremote... etc.), which means >> we can introduce some monitoring into our production environment and easily >> restart the JMX exporter as needed to iterate through configuration changes. >> > > The JMX exporter will pick up configuration changes without restarting > Very happy to hear that. I'll be sure to mention that in an upcoming doc PR, as that behaviour is not listed on the README.md that I could find. > [...] You also lose some process and JVM metrics. > I'm not sure that's necessarily the case; at least I can see MBeans for Memory, OperatingSytem, Runtime, Threading, etc. under the 'java.lang' domain, but that's likely to be JVM dependent, I don't think I saw that on Java 7 (or Tomcat 6). But I guess you mean that jmx_exporter is exporting some of its own information, rather than from MBeans for "process and JVM metrics" I would like to propose that we introduce one of two things: >> >> EITHER add a new attribute whitelistObjectNameAttributes that could be >> used for Jconsole-style attribute at a time (or similar; can you grab a few >> named attributes in one go?), which would allow for either the broad-brush >> or fine-brush approach to collecting the data; >> > > I'd be open to considering an attribute name blacklist, if there's a sane > way to specify that. > Looking at the following makes me think it should be reasonably straightforward to create an additional white/blacklists for Attributes within an ObjectName; I think the crux of it would be in located around this part of the code: https://github.com/prometheus/jmx_exporter/blob/master/collector/src/main/java/io/prometheus/jmx/JmxScraper.java#L140-L147 Map<String, MBeanAttributeInfo> name2AttrInfo = new LinkedHashMap<String, MBeanAttributeInfo>(); for (int idx = 0; idx < attrInfos.length; ++idx) { MBeanAttributeInfo attr = attrInfos[idx]; if (!attr.isReadable()) { logScrape(mbeanName, attr, "not readable"); continue; } name2AttrInfo.put(attr.getName(), attr); Plus there would have to the config-level stuff and validation thereof, and plumbing the config into the various method calls in places where where whitelistObjectName and blacklistObjectName appear. And test-cases. On a higher level, I'd suggest looking at ways to get metrics that don't > involve JMX. Using client_java directly as far as possible will avoid all > the fun and performance issues that JMX brings. > That would seem to be impractical in all my use-cases, as I don't control the application (certainly not the code-base) to that extent, and JMX would be the right, supported, tool for this job. I think I can probably work in the agent though; it seems to be the normal way for other similar vendors to do the same monitoring thing. -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/cda0dce5-603c-4af8-be79-712f7f2a6436o%40googlegroups.com.
