On Di, 09.06.26 11:25, Umut Tezduyar Lindskog ([email protected]) wrote: > Very nice work on the new report interface. I finally had a chance to play > with it in systemd 260 and 261. > > Ref: https://mastodon.social/@pid_eins/116220986239256436 > > systemd already sits on a lot of valuable host and service health metrics, > but unfortunately not many monitoring agents expose them comprehensively. > In the embedded space, for example, we find Fluent Bit compact enough for > our devices, but its systemd metrics support does not seem particularly > mature yet (at least as far as I know). Having a standard interface should > make it much easier to convince upstream agents to support these metrics > properly. > > As argued in the Mastodon post, the agent’s primary responsibility is to > collect and forward data. What are the upstream thoughts on offloading the > agent role entirely to systemd-report?
systemd-report is already capable of uploading the reports to a web service of your choice, in v261. The format is just JSON, in the format defined by the systemd metrics stuff. > OpenTelemetry has become a widely adopted standard. One challenge, however, > is that the OpenTelemetry metrics model is generally push-oriented, where > services export metrics to a collector (local or remote). Retrofitting such > a model onto some systemd metric providers, especially one-shot units, > could be fairly complicated. > > systemd-report, on the other hand, is already in a position to aggregate > metrics via Varlink. It could potentially act as the agent itself: collect > report metrics from providers and export them using OTLP (the OpenTelemetry > Protocol) to an OpenTelemetry collector. systemd's metrics format is on purpose kept conceptually close to the stuff Prometheus wants (though not in literal bytes), so that people can expose things reasonably that way across the network. However, to my knowledge noone actually did that work to implement such a bridge between systemd style metrics in other formats, at this time. Here at Amutable we are compiling reports regularly on nodes, and then upload them via the systemd-report native logic to a central server via HTTP, where we consume them, aggregate them. > More generally, I am curious about the longer-term vision here. Is there > interest upstream in moving beyond traditional monitoring agents, and what > are the current thoughts on OpenTelemetry integration? So, I think such a bridge would make a ton of sense, but at this point we are somewhat reluctant to adding new network-facing stuff in C. i.e. web-facing stuff really should probably be in rust. A very similar case for us I guess is the Varlink/HTTP proxy, which is implemented in Rust, and while under the umbrella of the systemd project is (currently) not part of the systemd tarball. One could even say that the varlink bridge is the write side and the report bridge would be the read side of things. Lennart -- Lennart Poettering, Berlin
