Hello Jean, Regarding Prometheus setup, we have implemented an optional HTTP endpoint. I would recommend you to include the following lines in your webadmin.properties:
extensions.routes=org.apache.james.webadmin.dropwizard.MetricsRoutes We should then: - better document this setup - (provide a docker-compose?) - Update the metric boards in /grafana-reporting Le 12/03/2021 à 21:06, Jean Helou a écrit : > Hello all, > > I agree that ES is not the best tool for metrics but it's implemented, > documented and demonstrated.... > - https://james.apache.org/server/metrics.html > - https://james.apache.org/server/config-elasticsearch.html > - > https://github.com/apache/james-project/blob/master/server/container/guice/cassandra-guice/src/main/java/org/apache/james/CassandraJamesServerMain.java#L141 > and that's only part of the mentions made throughout the documentation > > So today it's the most discoverable[1] way to collect metrics for james > especially for people who are not expert "ops". If this is removed, I feel > documenting at least one recommended way to do it is necessary. Let's document alternatives to the elasticsearch metric reporter. And mark the ElasticSearch reporter as legacy. The cost of maintaining two distinct ElasticSearch clusters will likely invite people to do the switch anyway... Would you agree with such an approach? > Having the metrics output to logs doesn't help much because of the amount > of processing required to get anything useful out of it (or I failed to > find how to easily ingest it please correct me) Logs for metrics had been introduced by Matthieu to get metrics out in the logs of a performance test platform, without the need to analyses the metrics "live". > which leaves JMX. > > Since JMX is a JVM standard, people who know the JVM well enough will > either know or easily deduce that there is a generic way to export JMX > metrics to whatever tool they want. However a mention of jmx_exporter[2] > for prometheus and jmxtrans [3] for other tools in the monitoring/jmx > sections of the documentation would be kind to non-jvm people who stumble > on apache james. Ideally having a demo setup/tutorial would be nice [4] We are moving away from JMX for the CLI, I see little reason to encourage its use here too. See https://www.cvedetails.com/cve/CVE-2017-12628/ > > One other thing regarding monitoring is that the README mentions Glowroot > [5] but there is no additional mention in the documentation. I am not > an ops expert so I can't help but wonder how that fits with the ES exported > metric ... > The glowroot documentation mentions a lot of features some of which overlap > with prometheus. The project doesn't seem very mature: even if it was > started 6 years ago, there have been very few contributions in the last > year, the highest released version is 0.13.6 (james uses 0.13.4) and the > documentation is quite sparse. I am not sure this is a good recommendation > to put in the readme of the james project. Glowroot is an APM, allowing to : - capture slow traces - capture DB queries (globally and on slow traces) - do some flame graphs captures (that might be more or less relevant based on the number of captures performed) It provides some insight that regular monitoring tools will not give. I would categorize it more as a developer diagnosis tool (which to me is invaluable for performance issues). It do not replace a proper monitoring tool. > > Jean > > [1] > https://www.google.com/search?q=monitoring+apache+james+metrics&oq=monitoring+apache+james+metrics > [2] https://github.com/prometheus/jmx_exporter > [3] https://github.com/jmxtrans/jmxtrans > [4] on that note, one reason I have been very quiet lately is that I am > working on a helm chart and a terraform recipe to deploy james to a k8s > cluster including all the associated software dependencies. I'm getting > there and I intend to contribute this back to the project. One of the > things I don't have yet is ... monitoring ;) Since k8s' monitoring tool of > choice is prometheus that's what I intended to use but I never used it > before and I'm only a casual contributor so I progress slowly That looks cool! > [5] https://glowroot.org/ > > > On Thu, Mar 11, 2021 at 2:35 PM Juhan Aasaru <[email protected]> wrote: > >> If the community agrees then I propose the following. >> >> * we deprecate this functionality in the next release (3.6.0) with a note >> that it would be completely removed in the next release (3.7.0) >> * this functionality (James storing history of its metrics to >> Elasticsearch) would continue to work with Elasticsearch 6 while rest of >> the Elasticsearch-related functionality would be upgraded to Elasticsearch >> 7 >> * anyone still using it would have to run two different Elasticsearch >> instances (Elasticsearch 6 for history of metrics and Elasticsearch 7 for >> rest of James) >> >> Juhan >> >> >> >> Kontakt Juhan Aasaru (<[email protected]>) kirjutas kuupäeval N, 11. märts >> 2021 kell 15:17: >> >>> Hi! >>> >>> There is currently ongoing work to upgrade James to Elasticsearch 7.x >>> See https://github.com/apache/james-project/pull/328 >>> >>> Current James can be configured to save James metrics to a separate index >>> in Elasticsearch. >>> And then Grafana dashboards can be configured to display the history of >>> these saved metrics. >>> For more detailed documentation refer the configuration parameter >>> "elasticsearch.metrics.reports.enabled" here: >>> https://james.apache.org/server/config-elasticsearch.html >>> >>> This functionality that gathers and publish metrics is provided by an >>> unmaintained library: >>> https://github.com/linagora/elasticsearch-metrics-reporter-java >>> This library is not compatible with Elasticsearch 7.x >>> >>> We are proposing to remove this functionality from James as >>> the industry standard is to use external tools that are purpose built for >>> pulling and storing the metrics. >>> >>> Users currently relying on this functionality would have to configure >> some >>> monitoring tool (like Prometheus) to regularly pull and store these >> metrics >>> if they want to continue displaying history of various James related >>> metrics over time. >>> >>> Please respond if you agree or are against removing it. >>> >>> Kind regards >>> Juhan Aasaru >>> >>> >> > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
