Hello all, I agree that ES is not the best tool for metrics but it's implemented, documented and demonstrated.... - https://james.apache.org/server/metrics.html - https://james.apache.org/server/config-elasticsearch.html - https://github.com/apache/james-project/blob/master/server/container/guice/cassandra-guice/src/main/java/org/apache/james/CassandraJamesServerMain.java#L141 and that's only part of the mentions made throughout the documentation
So today it's the most discoverable[1] way to collect metrics for james especially for people who are not expert "ops". If this is removed, I feel documenting at least one recommended way to do it is necessary. Having the metrics output to logs doesn't help much because of the amount of processing required to get anything useful out of it (or I failed to find how to easily ingest it please correct me) which leaves JMX. Since JMX is a JVM standard, people who know the JVM well enough will either know or easily deduce that there is a generic way to export JMX metrics to whatever tool they want. However a mention of jmx_exporter[2] for prometheus and jmxtrans [3] for other tools in the monitoring/jmx sections of the documentation would be kind to non-jvm people who stumble on apache james. Ideally having a demo setup/tutorial would be nice [4] One other thing regarding monitoring is that the README mentions Glowroot [5] but there is no additional mention in the documentation. I am not an ops expert so I can't help but wonder how that fits with the ES exported metric ... The glowroot documentation mentions a lot of features some of which overlap with prometheus. The project doesn't seem very mature: even if it was started 6 years ago, there have been very few contributions in the last year, the highest released version is 0.13.6 (james uses 0.13.4) and the documentation is quite sparse. I am not sure this is a good recommendation to put in the readme of the james project. Jean [1] https://www.google.com/search?q=monitoring+apache+james+metrics&oq=monitoring+apache+james+metrics [2] https://github.com/prometheus/jmx_exporter [3] https://github.com/jmxtrans/jmxtrans [4] on that note, one reason I have been very quiet lately is that I am working on a helm chart and a terraform recipe to deploy james to a k8s cluster including all the associated software dependencies. I'm getting there and I intend to contribute this back to the project. One of the things I don't have yet is ... monitoring ;) Since k8s' monitoring tool of choice is prometheus that's what I intended to use but I never used it before and I'm only a casual contributor so I progress slowly [5] https://glowroot.org/ On Thu, Mar 11, 2021 at 2:35 PM Juhan Aasaru <[email protected]> wrote: > If the community agrees then I propose the following. > > * we deprecate this functionality in the next release (3.6.0) with a note > that it would be completely removed in the next release (3.7.0) > * this functionality (James storing history of its metrics to > Elasticsearch) would continue to work with Elasticsearch 6 while rest of > the Elasticsearch-related functionality would be upgraded to Elasticsearch > 7 > * anyone still using it would have to run two different Elasticsearch > instances (Elasticsearch 6 for history of metrics and Elasticsearch 7 for > rest of James) > > Juhan > > > > Kontakt Juhan Aasaru (<[email protected]>) kirjutas kuupäeval N, 11. märts > 2021 kell 15:17: > > > Hi! > > > > There is currently ongoing work to upgrade James to Elasticsearch 7.x > > See https://github.com/apache/james-project/pull/328 > > > > Current James can be configured to save James metrics to a separate index > > in Elasticsearch. > > And then Grafana dashboards can be configured to display the history of > > these saved metrics. > > For more detailed documentation refer the configuration parameter > > "elasticsearch.metrics.reports.enabled" here: > > https://james.apache.org/server/config-elasticsearch.html > > > > This functionality that gathers and publish metrics is provided by an > > unmaintained library: > > https://github.com/linagora/elasticsearch-metrics-reporter-java > > This library is not compatible with Elasticsearch 7.x > > > > We are proposing to remove this functionality from James as > > the industry standard is to use external tools that are purpose built for > > pulling and storing the metrics. > > > > Users currently relying on this functionality would have to configure > some > > monitoring tool (like Prometheus) to regularly pull and store these > metrics > > if they want to continue displaying history of various James related > > metrics over time. > > > > Please respond if you agree or are against removing it. > > > > Kind regards > > Juhan Aasaru > > > > >
