Re: [gentoo-user] logs in the browser?
On 24.02.2015 17:01, Rich Freeman wrote: Seems like there should be a systemd-users mailing list, actually. This sort of situation is completely distro-agnostic. Yes! And systemd-devel ml is always kind of they will laugh at me and say ugly things! ;-) You certainly could design such an application. If you do so I'd consider pulling the journald logs in JSON format. I'd also see if somebody actually has written a journald library/class/etc for your language of choice - it seems like that is the sort of thing that is likely to exist soon if not already. One of the goals journald (and systemd) is to provide more of an API for services so that there is less parsing of text files and communicating via signals/etc. I'm sure with appropriate permissions a process could just obtain log entries via dbus, and using cursors poll for new entries (or maybe there is a push/stream mechanism). Really though it seems like the solution is a generic log monitor with rules for such things, with the monitor utilizing the JSON data from journald for richer metadata/efficiency/accuracy. Well, it might be a nice challenge to do so ... sure! The real world challenge is that I have to provide the postfix-logs for one domain on one server (so far) for one admin to browse through (without giving him ssh-access ... it's a noob ...) No overkill paid here. But maybe I ask on systemd-devel-ml ... Stefan
Re: [gentoo-user] logs in the browser?
On Tue, Feb 24, 2015 at 3:27 PM, Stefan G. Weichinger li...@xunil.at wrote: On 24.02.2015 17:01, Rich Freeman wrote: Seems like there should be a systemd-users mailing list, actually. This sort of situation is completely distro-agnostic. Yes! And systemd-devel ml is always kind of they will laugh at me and say ugly things! ;-) Honestly, I've never received treatment like this on the systemd lists or irc channel. However, when I post there it tends to be about specific bugs with details/etc. You're not really talking about systemd development, but rather questions around best practices for using it, discussion, etc. It would be like posting this thread on gentoo-dev - it isn't the purpose of the list, but such discussion is completely welcome here. If we were discussing an eclass change, on the other hand, gentoo-dev would be the right place. You might try asking on #systemd. That doesn't really have a defined topic afaict, and discussion is a lot more free. Of course, the downside to irc is you only hear from people online at the moment. You might ask about a user mailing list. -- Rich
Re: [gentoo-user] logs in the browser?
Am 24.02.2015 um 03:14 schrieb Canek Peláez Valdés: Stefan, if you already have systemd (which I believe you do), why don't you compile in the support for microhttpd and use the journal? This is the exact scenario for which systemd-journal-gatewayd[1] was written. very good ... enabled it on one of my machines, looks good. I am unsure if it will be possible to see only postfix.service (easier) AND only the lines relevant for the domain of the specific customer by doing this? thanks, Stefan
Re: [gentoo-user] logs in the browser?
On Tue, Feb 24, 2015 at 4:50 AM, Stefan G. Weichinger li...@xunil.at wrote: Am 24.02.2015 um 03:14 schrieb Canek Peláez Valdés: Stefan, if you already have systemd (which I believe you do), why don't you compile in the support for microhttpd and use the journal? This is the exact scenario for which systemd-journal-gatewayd[1] was written. very good ... enabled it on one of my machines, looks good. I am unsure if it will be possible to see only postfix.service (easier) I suspect this is trivial - it looks like something like this would work: http://.../entries?_SYSTEMD_UNIT=postfix.service (note, you might need to tweak that - I haven't used the http gateway personally and am going from the manpage) AND only the lines relevant for the domain of the specific customer by doing this? I think you're going to be stuck here unless they come from different machines or something like that. Obviously you can pipe the output through grep but journald will only pre-filter the output using journal fields, like facility, priority, etc. syslog only provided a few fields for clients to specify, and this is probably because in the end the data just got dumped to a text file so that it wasn't searchable by field anyway. It would be nice if they extended the syslog protocol for systemd and made it possible for clients to specify additional fields, but obviously the client would need to support this (likely sending logs over dbus or such). The http gateway seems like it is intended more as a transport mechanism with some usability for ad-hoc human viewing. It isn't a full-fledged log analysis tool. The fact that journald can output in JSON with uuids for each entry should make it far easier to parse its logs with an analysis tool, but I think all those vendors are playing catch-up. I suspect they'll support it fairly soon once they see everybody using it. From a machine parsing standpoint the fielded binary format makes a lot more sense. -- Rich
Re: [gentoo-user] logs in the browser?
On Tue, Feb 24, 2015 at 10:33 AM, Stefan G. Weichinger li...@xunil.at wrote: [ ... ] Maybe I could set up some other web-app that (a) looks at the link pointing to the postfix.service-logs and (b) filters them? (With my programmer's hat on): I think the easiest way would be to create a little client that downloads the logs in JSON format, and then do the filtering off-site. If I'm not mistaken, libmicrohttpd supports Accept-Encoding: gzip, and therefore the used bandwidth should not be a lot. Also, you can get the last cursor from the journal the first time, and next time you download logs, you start from that cursor on, so you don't download everything again. I don't see many advantages on doing the filtering on-site. Specially if, after a while, you are handling several servers. Regards. -- Canek Peláez Valdés Profesor de asignatura, Facultad de Ciencias Universidad Nacional Autónoma de México
Re: [gentoo-user] logs in the browser?
On 24.02.2015 13:14, Rich Freeman wrote: I suspect this is trivial - it looks like something like this would work: http://.../entries?_SYSTEMD_UNIT=postfix.service Yes, correct, as I thought this is the easy part. Works: http://mythtv.local:19531/entries?_SYSTEMD_UNIT=postfix.service (using my mythbackend as test target). (note, you might need to tweak that - I haven't used the http gateway personally and am going from the manpage) AND only the lines relevant for the domain of the specific customer by doing this? I think you're going to be stuck here unless they come from different machines or something like that. Obviously you can pipe the output through grep but journald will only pre-filter the output using journal fields, like facility, priority, etc. syslog only provided a few fields for clients to specify, and this is probably because in the end the data just got dumped to a text file so that it wasn't searchable by field anyway. It would be nice if they extended the syslog protocol for systemd and made it possible for clients to specify additional fields, but obviously the client would need to support this (likely sending logs over dbus or such). The http gateway seems like it is intended more as a transport mechanism with some usability for ad-hoc human viewing. It isn't a full-fledged log analysis tool. The fact that journald can output in JSON with uuids for each entry should make it far easier to parse its logs with an analysis tool, but I think all those vendors are playing catch-up. I suspect they'll support it fairly soon once they see everybody using it. From a machine parsing standpoint the fielded binary format makes a lot more sense. Maybe I could set up some other web-app that (a) looks at the link pointing to the postfix.service-logs and (b) filters them? I could post to the systemd-devel-ml ... btw ;-) Stefan
Re: [gentoo-user] logs in the browser?
On Tue, Feb 24, 2015 at 10:33 AM, Stefan G. Weichinger li...@xunil.at wrote: Maybe I could set up some other web-app that (a) looks at the link pointing to the postfix.service-logs and (b) filters them? I could post to the systemd-devel-ml ... btw ;-) Seems like there should be a systemd-users mailing list, actually. This sort of situation is completely distro-agnostic. You certainly could design such an application. If you do so I'd consider pulling the journald logs in JSON format. I'd also see if somebody actually has written a journald library/class/etc for your language of choice - it seems like that is the sort of thing that is likely to exist soon if not already. One of the goals journald (and systemd) is to provide more of an API for services so that there is less parsing of text files and communicating via signals/etc. I'm sure with appropriate permissions a process could just obtain log entries via dbus, and using cursors poll for new entries (or maybe there is a push/stream mechanism). Really though it seems like the solution is a generic log monitor with rules for such things, with the monitor utilizing the JSON data from journald for richer metadata/efficiency/accuracy. -- Rich
[gentoo-user] logs in the browser?
As I see that syslog-thread ... I think of setting up something for apache that allows a client to browse and search through the postfix-logs for his domain. Any good hints what to use for that purpose? Stefan
Re: [gentoo-user] logs in the browser?
On Mon, Feb 23, 2015 at 4:20 PM, Stefan G. Weichinger li...@xunil.at wrote: As I see that syslog-thread ... I think of setting up something for apache that allows a client to browse and search through the postfix-logs for his domain. Any good hints what to use for that purpose? Stefan, if you already have systemd (which I believe you do), why don't you compile in the support for microhttpd and use the journal? This is the exact scenario for which systemd-journal-gatewayd[1] was written. Regards. [1] http://www.freedesktop.org/software/systemd/man/systemd-journal-gatewayd.service.html -- Canek Peláez Valdés Profesor de asignatura, Facultad de Ciencias Universidad Nacional Autónoma de México
