[systemd-devel] sequence numbers of udev events not in sequence
Hi! I happened to be printing out the sequence numbers of the udev events I was receiving via the udev monitor, and I noticed that they did not occur exactly in sequence, e.g., I was receiving events with sequence numbers in this order: 13694 13696 13695 13697 13698 13699 13700 13701 13702 13703 13706 13704 13705 How should I interpret this? Is it behavior I should expect? Thanks for any help. - mulhern ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] What is udev_enumerate_scan_subsystems actually supposed to be doing?
I also noticed that the directories yielded by using udev_enumerate_scan_subsystems do not actually include, using a pretty loose meaning of include, every subsystem that udevadm info --export-db considers at least one device to belong to. So, I'm really not sure what these values that it yields represent at all. - mulhern - Original Message - > From: "Anne Mulhern" <amulh...@redhat.com> > To: "systemd" <systemd-devel@lists.freedesktop.org> > Sent: Monday, June 13, 2016 5:53:30 PM > Subject: [systemd-devel] What is udev_enumerate_scan_subsystems actually > supposed to be doing? > > I'm asking as the maintainer of pyudev, trying to come up with a design > in pyudev that reflects that actual purpose and function of this procedure. > > I can see that it gives me a list of directories, some in /sys/bus and > some in /sys/module. I can tell that it takes as an argument a pointer to > a udev_enumerate struct, but I'm pretty sure that most of the procedures > that also take a udev_enumerate struct would be pretty nonsensical to use > if the purpose was eventually to call udev_enumerate_scan_subsystems on the > udev_enumerate struct. I can tell that the list of items returned by > udev_enumerate_scan_subsystems is a lot larger than the set of subsystems > to which devices belong according to udevadm info --export-db. > > So, what is the actual meaning of these directory names returned when > the struct is traversed after calling udev_enumerate_scan_subsystems? > > Thanks for your help, > > - mulhern > ___ > systemd-devel mailing list > systemd-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/systemd-devel > ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] What is udev_enumerate_scan_subsystems actually supposed to be doing?
I'm asking as the maintainer of pyudev, trying to come up with a design in pyudev that reflects that actual purpose and function of this procedure. I can see that it gives me a list of directories, some in /sys/bus and some in /sys/module. I can tell that it takes as an argument a pointer to a udev_enumerate struct, but I'm pretty sure that most of the procedures that also take a udev_enumerate struct would be pretty nonsensical to use if the purpose was eventually to call udev_enumerate_scan_subsystems on the udev_enumerate struct. I can tell that the list of items returned by udev_enumerate_scan_subsystems is a lot larger than the set of subsystems to which devices belong according to udevadm info --export-db. So, what is the actual meaning of these directory names returned when the struct is traversed after calling udev_enumerate_scan_subsystems? Thanks for your help, - mulhern ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] Any generic udev method for handling missing external programs?
Hi! I wondered if there already exists or were plans for some generic way to have an error on missing external programs that are invoked by udev rules. My particular problem was that somehow kpartx had gone missing on my system. 62-multipath-rules had a rule for invoking kpartx in order to make some linear devices to map to partition devices. It invoked these, and the call failed. This failure was logged in the journal, at level 3, which seems reasonable. However, there was no connection of the journal log entry with the multipath rule, or anything like that. The error was reported by the spawn_exec method, thus: """ Tue 2016-03-29 13:44:59.614256 EDT [s=07d0dd3e450e447b8176fae6f6ca2172;i=2759c4;b=4a123904d9fd4e2 7883b8169b32e4f8b;m=1b70a1aa78c;t=52f3396931d23;x=9bc01c815423a6b0] SYSLOG_FACILITY=3 _TRANSPORT=journal _UID=0 _GID=0 _CAP_EFFECTIVE=1f _BOOT_ID=4a123904d9fd4e27883b8169b32e4f8b _MACHINE_ID=c2a1a2c9555048fab5453a08361c43dc _HOSTNAME=megadeth.lab.bos.redhat.com PRIORITY=3 SYSLOG_IDENTIFIER=systemd-udevd _COMM=systemd-udevd CODE_FILE=src/udev/udev-event.c CODE_LINE=412 CODE_FUNCTION=spawn_exec ERRNO=2 MESSAGE=failed to execute '/sbin/kpartx' '/sbin/kpartx -a /dev/dm-25': No such file or directory _PID=40023 _SOURCE_REALTIME_TIMESTAMP=1459273499614256 """ The problem ramified, since the missing multipath partitions were part of another device, and so forth. It seems like it would be nice for the udev rules to be able to specify their external dependencies, and if their external dependencies were lacking, to stow a more useful message in the journal at some point. If this can't be checked until about to execute, then it would be nice if there was a somewhat more helpful message then the current one, which just says that the file is not found. If it can be checked when the rules are being processed, then udev could log the problem at that time. An argument could be made that no external programs that aren't certain to be installed should be called in the udev rules. kpartx is required by multipath, so in theory this rule would have ensured correctness. But, accidents do happen, like in this case where kpartx package was installed but kpartx executable had left the building. (Actually, kpartx package appears to be required by systemd so it would be reasonable to expect the executable to be around.) So, is it impossible, likely to happen soon, already done, not worth the trouble? TIA, - mulhern ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] libudev: subdirectories in sysfs (what does "available" mean?)
- Original Message - > From: "Greg KH" <gre...@linuxfoundation.org> > To: "Anne Mulhern" <amulh...@redhat.com> > Cc: "David Herrmann" <dh.herrm...@gmail.com>, "systemd" > <systemd-devel@lists.freedesktop.org> > Sent: Tuesday, November 24, 2015 3:15:21 PM > Subject: Re: [systemd-devel] libudev: subdirectories in sysfs (what does > "available" mean?) > > On Tue, Nov 24, 2015 at 12:37:46PM -0500, Anne Mulhern wrote: > > > > - Original Message - > > > From: "Greg KH" <gre...@linuxfoundation.org> > > > To: "Anne Mulhern" <amulh...@redhat.com> > > > Cc: "David Herrmann" <dh.herrm...@gmail.com>, "systemd" > > > <systemd-devel@lists.freedesktop.org> > > > Sent: Tuesday, November 24, 2015 11:42:21 AM > > > Subject: Re: [systemd-devel] libudev: subdirectories in sysfs (what does > > > "available" mean?) > > > > > > On Tue, Nov 24, 2015 at 10:37:08AM -0500, Anne Mulhern wrote: > > > > > > > > > > > > > > > > > > > > - Original Message - > > > > > From: "David Herrmann" <dh.herrm...@gmail.com> > > > > > To: "Anne Mulhern" <amulh...@redhat.com> > > > > > Cc: "systemd" <systemd-devel@lists.freedesktop.org> > > > > > Sent: Tuesday, November 24, 2015 10:15:05 AM > > > > > Subject: Re: [systemd-devel] libudev: subdirectories in sysfs (what > > > > > does > > > > > "available" mean?) > > > > > > > > > > Hi > > > > > > > > > > On Tue, Nov 24, 2015 at 3:54 PM, Anne Mulhern <amulh...@redhat.com> > > > > > wrote: > > > > > >> From: "David Herrmann" <dh.herrm...@gmail.com> > > > > > >> On Tue, Nov 17, 2015 at 11:57 PM, Anne Mulhern > > > > > >> <amulh...@redhat.com> > > > > > >> wrote: > > > > > >> >> From: "David Herrmann" <dh.herrm...@gmail.com> > > > > > >> >> On Mon, Nov 16, 2015 at 5:35 PM, Anne Mulhern > > > > > >> >> <amulh...@redhat.com> > > > > > >> >> wrote: > > > > > >> >> > libudev has some cooperating procedures that return the keys > > > > > >> >> > for > > > > > >> >> > a > > > > > >> >> > bunch > > > > > >> >> > of > > > > > >> >> > sysfs attributes for a given device. > > > > > >> >> > > > > > > >> >> > These attributes all correspond to files that are stored in > > > > > >> >> > the > > > > > >> >> > sysfs > > > > > >> >> > device directory. > > > > > >> >> > > > > > > >> >> > In the same directory there are sometimes subdirectories, > > > > > >> >> > that > > > > > >> >> > themselves > > > > > >> >> > contain files > > > > > >> >> > with information about their corresponding attribute. The dm > > > > > >> >> > directory > > > > > >> >> > is > > > > > >> >> > one obvious > > > > > >> >> > example. > > > > > >> >> > > > > > > >> >> > Are their any plans for libudev to add an ability to get the > > > > > >> >> > values > > > > > >> >> > from > > > > > >> >> > these subdirectories > > > > > >> >> > as some kind of attributes? > > > > > >> >> > > > > > > >> >> > If no, why? > > > > > >> >> > > > > > >> >> sd_device_get_sysattr_value(device, "foo/bar/baz", ); > > > > > >> >> > > > > > >> >> This should work fine (or its udev_device_* equivalent). > > > > > >> >> > > > > > >> >> Btw., I recommend just using readdir(), open(), read(), and > > >
Re: [systemd-devel] libudev: subdirectories in sysfs (what does "available" mean?)
- Original Message - > From: "Greg KH" <gre...@linuxfoundation.org> > To: "Anne Mulhern" <amulh...@redhat.com> > Cc: "David Herrmann" <dh.herrm...@gmail.com>, "systemd" > <systemd-devel@lists.freedesktop.org> > Sent: Tuesday, November 24, 2015 11:42:21 AM > Subject: Re: [systemd-devel] libudev: subdirectories in sysfs (what does > "available" mean?) > > On Tue, Nov 24, 2015 at 10:37:08AM -0500, Anne Mulhern wrote: > > > > > > > > > > - Original Message - > > > From: "David Herrmann" <dh.herrm...@gmail.com> > > > To: "Anne Mulhern" <amulh...@redhat.com> > > > Cc: "systemd" <systemd-devel@lists.freedesktop.org> > > > Sent: Tuesday, November 24, 2015 10:15:05 AM > > > Subject: Re: [systemd-devel] libudev: subdirectories in sysfs (what does > > > "available" mean?) > > > > > > Hi > > > > > > On Tue, Nov 24, 2015 at 3:54 PM, Anne Mulhern <amulh...@redhat.com> > > > wrote: > > > >> From: "David Herrmann" <dh.herrm...@gmail.com> > > > >> On Tue, Nov 17, 2015 at 11:57 PM, Anne Mulhern <amulh...@redhat.com> > > > >> wrote: > > > >> >> From: "David Herrmann" <dh.herrm...@gmail.com> > > > >> >> On Mon, Nov 16, 2015 at 5:35 PM, Anne Mulhern <amulh...@redhat.com> > > > >> >> wrote: > > > >> >> > libudev has some cooperating procedures that return the keys for > > > >> >> > a > > > >> >> > bunch > > > >> >> > of > > > >> >> > sysfs attributes for a given device. > > > >> >> > > > > >> >> > These attributes all correspond to files that are stored in the > > > >> >> > sysfs > > > >> >> > device directory. > > > >> >> > > > > >> >> > In the same directory there are sometimes subdirectories, that > > > >> >> > themselves > > > >> >> > contain files > > > >> >> > with information about their corresponding attribute. The dm > > > >> >> > directory > > > >> >> > is > > > >> >> > one obvious > > > >> >> > example. > > > >> >> > > > > >> >> > Are their any plans for libudev to add an ability to get the > > > >> >> > values > > > >> >> > from > > > >> >> > these subdirectories > > > >> >> > as some kind of attributes? > > > >> >> > > > > >> >> > If no, why? > > > >> >> > > > >> >> sd_device_get_sysattr_value(device, "foo/bar/baz", ); > > > >> >> > > > >> >> This should work fine (or its udev_device_* equivalent). > > > >> >> > > > >> >> Btw., I recommend just using readdir(), open(), read(), and > > > >> >> write(). > > > >> >> sysfs is a filesystem, no reason to wrap all those commands. > > > >> > > > > >> > Thanks, I'm asking this more as the pyudev maintainer than as > > > >> > someone > > > >> > who actually wants these values. > > > >> > > > > >> > The funny thing is, I recently found out that the list obtained by > > > >> > udev_device_get_sysattr_list_entry () and friends contains so > > > >> > called "available" keys, but when those get passed to > > > >> > udev_device_get_sysattr_value () the result might be NULL. > > > >> > That makes sense in the sense that they might represent files > > > >> > that are unreadable. > > > >> > > > > >> > Now I find out that I can make up keys not in the results of > > > >> > udev_device_get_sysattr_list_entry () and pass those to > > > >> > udev_device_get_sysattr_value() and get a non-null result. > > > >> > > > > >> > So, what does "available" mean? Do these sysattr_list_entry() > > > >> > methods give any useful information? > > > >> > > > >> "available" p
Re: [systemd-devel] libudev: subdirectories in sysfs (what does "available" mean?)
- Original Message - > From: "David Herrmann" <dh.herrm...@gmail.com> > To: "Anne Mulhern" <amulh...@redhat.com> > Cc: "systemd" <systemd-devel@lists.freedesktop.org> > Sent: Tuesday, November 24, 2015 4:43:24 AM > Subject: Re: [systemd-devel] libudev: subdirectories in sysfs (what does > "available" mean?) > > Hi > > On Tue, Nov 17, 2015 at 11:57 PM, Anne Mulhern <amulh...@redhat.com> wrote: > >> From: "David Herrmann" <dh.herrm...@gmail.com> > >> On Mon, Nov 16, 2015 at 5:35 PM, Anne Mulhern <amulh...@redhat.com> wrote: > >> > libudev has some cooperating procedures that return the keys for a bunch > >> > of > >> > sysfs attributes for a given device. > >> > > >> > These attributes all correspond to files that are stored in the sysfs > >> > device directory. > >> > > >> > In the same directory there are sometimes subdirectories, that > >> > themselves > >> > contain files > >> > with information about their corresponding attribute. The dm directory > >> > is > >> > one obvious > >> > example. > >> > > >> > Are their any plans for libudev to add an ability to get the values from > >> > these subdirectories > >> > as some kind of attributes? > >> > > >> > If no, why? > >> > >> sd_device_get_sysattr_value(device, "foo/bar/baz", ); > >> > >> This should work fine (or its udev_device_* equivalent). > >> > >> Btw., I recommend just using readdir(), open(), read(), and write(). > >> sysfs is a filesystem, no reason to wrap all those commands. > > > > Thanks, I'm asking this more as the pyudev maintainer than as someone > > who actually wants these values. > > > > The funny thing is, I recently found out that the list obtained by > > udev_device_get_sysattr_list_entry () and friends contains so > > called "available" keys, but when those get passed to > > udev_device_get_sysattr_value () the result might be NULL. > > That makes sense in the sense that they might represent files > > that are unreadable. > > > > Now I find out that I can make up keys not in the results of > > udev_device_get_sysattr_list_entry () and pass those to > > udev_device_get_sysattr_value() and get a non-null result. > > > > So, what does "available" mean? Do these sysattr_list_entry() > > methods give any useful information? > > "available" probably means attributes which are direct descendants of > the device. That is, sysattr_list_entry() only lists such direct > descendants, while sysattr_value() allows you to query anything (you > probably can even pass "foo/../../bar/baz"). > > Thanks > David > I think it's yet more complicated than that. For example, 'dm' (for device mapper) is not in the list of available attributes, but 'dm/name' is certainly an attribute that can be read by sysattr_value(). - mulhern ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] libudev: subdirectories in sysfs (what does "available" mean?)
- Original Message - > From: "David Herrmann" <dh.herrm...@gmail.com> > To: "Anne Mulhern" <amulh...@redhat.com> > Cc: "systemd" <systemd-devel@lists.freedesktop.org> > Sent: Tuesday, November 24, 2015 10:15:05 AM > Subject: Re: [systemd-devel] libudev: subdirectories in sysfs (what does > "available" mean?) > > Hi > > On Tue, Nov 24, 2015 at 3:54 PM, Anne Mulhern <amulh...@redhat.com> wrote: > >> From: "David Herrmann" <dh.herrm...@gmail.com> > >> On Tue, Nov 17, 2015 at 11:57 PM, Anne Mulhern <amulh...@redhat.com> > >> wrote: > >> >> From: "David Herrmann" <dh.herrm...@gmail.com> > >> >> On Mon, Nov 16, 2015 at 5:35 PM, Anne Mulhern <amulh...@redhat.com> > >> >> wrote: > >> >> > libudev has some cooperating procedures that return the keys for a > >> >> > bunch > >> >> > of > >> >> > sysfs attributes for a given device. > >> >> > > >> >> > These attributes all correspond to files that are stored in the sysfs > >> >> > device directory. > >> >> > > >> >> > In the same directory there are sometimes subdirectories, that > >> >> > themselves > >> >> > contain files > >> >> > with information about their corresponding attribute. The dm > >> >> > directory > >> >> > is > >> >> > one obvious > >> >> > example. > >> >> > > >> >> > Are their any plans for libudev to add an ability to get the values > >> >> > from > >> >> > these subdirectories > >> >> > as some kind of attributes? > >> >> > > >> >> > If no, why? > >> >> > >> >> sd_device_get_sysattr_value(device, "foo/bar/baz", ); > >> >> > >> >> This should work fine (or its udev_device_* equivalent). > >> >> > >> >> Btw., I recommend just using readdir(), open(), read(), and write(). > >> >> sysfs is a filesystem, no reason to wrap all those commands. > >> > > >> > Thanks, I'm asking this more as the pyudev maintainer than as someone > >> > who actually wants these values. > >> > > >> > The funny thing is, I recently found out that the list obtained by > >> > udev_device_get_sysattr_list_entry () and friends contains so > >> > called "available" keys, but when those get passed to > >> > udev_device_get_sysattr_value () the result might be NULL. > >> > That makes sense in the sense that they might represent files > >> > that are unreadable. > >> > > >> > Now I find out that I can make up keys not in the results of > >> > udev_device_get_sysattr_list_entry () and pass those to > >> > udev_device_get_sysattr_value() and get a non-null result. > >> > > >> > So, what does "available" mean? Do these sysattr_list_entry() > >> > methods give any useful information? > >> > >> "available" probably means attributes which are direct descendants of > >> the device. That is, sysattr_list_entry() only lists such direct > >> descendants, while sysattr_value() allows you to query anything (you > >> probably can even pass "foo/../../bar/baz"). > >> > >> Thanks > >> David > >> > > > > I think it's yet more complicated than that. For example, > > 'dm' (for device mapper) is not in the list of available > > attributes, but 'dm/name' is certainly an attribute that > > can be read by sysattr_value(). > > 'dm' is not an attribute, so it will never be listed as available > attribute. Directories are never treated as attributes. > > Thanks > David > But "bdi" is listed as an attribute, and is a directory. - mulhern ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] libudev: subdirectories in sysfs (what does "available" mean?)
- Original Message - > From: "David Herrmann" <dh.herrm...@gmail.com> > To: "Anne Mulhern" <amulh...@redhat.com> > Cc: "systemd" <systemd-devel@lists.freedesktop.org> > Sent: Tuesday, November 17, 2015 10:48:19 AM > Subject: Re: [systemd-devel] libudev: subdirectories in sysfs > > Hi > > On Mon, Nov 16, 2015 at 5:35 PM, Anne Mulhern <amulh...@redhat.com> wrote: > > Hi! > > > > libudev has some cooperating procedures that return the keys for a bunch of > > sysfs attributes for a given device. > > > > These attributes all correspond to files that are stored in the sysfs > > device directory. > > > > In the same directory there are sometimes subdirectories, that themselves > > contain files > > with information about their corresponding attribute. The dm directory is > > one obvious > > example. > > > > Are their any plans for libudev to add an ability to get the values from > > these subdirectories > > as some kind of attributes? > > > > If no, why? > > sd_device_get_sysattr_value(device, "foo/bar/baz", ); > > This should work fine (or its udev_device_* equivalent). > > Btw., I recommend just using readdir(), open(), read(), and write(). > sysfs is a filesystem, no reason to wrap all those commands. > > Thanks > David > Thanks, I'm asking this more as the pyudev maintainer than as someone who actually wants these values. The funny thing is, I recently found out that the list obtained by udev_device_get_sysattr_list_entry () and friends contains so called "available" keys, but when those get passed to udev_device_get_sysattr_value () the result might be NULL. That makes sense in the sense that they might represent files that are unreadable. Now I find out that I can make up keys not in the results of udev_device_get_sysattr_list_entry () and pass those to udev_device_get_sysattr_value() and get a non-null result. So, what does "available" mean? Do these sysattr_list_entry() methods give any useful information? - mulhern ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] libudev: subdirectories in sysfs
Hi! libudev has some cooperating procedures that return the keys for a bunch of sysfs attributes for a given device. These attributes all correspond to files that are stored in the sysfs device directory. In the same directory there are sometimes subdirectories, that themselves contain files with information about their corresponding attribute. The dm directory is one obvious example. Are their any plans for libudev to add an ability to get the values from these subdirectories as some kind of attributes? If no, why? Thanks! - mulhern ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] Pretty sure that libudev does not provide a facility
for traversing sysfs directory via slaves or holders subdirectories. lsblk calls the holders direction inverse. python-pyudev does not provide such a facility, (but it unofficially discontinued development in 2013). And lsblk does not seem to use libudev facilities directly for this traversal. So, I think I'm probably right. Can someone confirm my supposition (or even better, contradict it)? Thanks! - mulhern ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] RFC: filter and search journalctl
- Original Message - From: Anne Mulhern amulh...@redhat.com To: systemd-devel@lists.freedesktop.org Sent: Monday, August 17, 2015 11:34:10 AM Subject: Re: [systemd-devel] RFC: filter and search journalctl - Original Message - From: Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl To: Anne Mulhern amulh...@redhat.com Cc: systemd-devel@lists.freedesktop.org, Sebastian Schindler sebastian.schind...@travelping.com Sent: Monday, August 17, 2015 10:45:11 AM Subject: Re: [systemd-devel] RFC: filter and search journalctl On Mon, Aug 17, 2015 at 10:24:22AM -0400, Anne Mulhern wrote: - Original Message - From: Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl To: Sebastian Schindler sebastian.schind...@travelping.com Cc: systemd-devel@lists.freedesktop.org Sent: Saturday, August 8, 2015 3:48:30 PM Subject: Re: [systemd-devel] RFC: filter and search journalctl On Fri, Aug 07, 2015 at 11:53:13AM +0200, Sebastian Schindler wrote: Grep-ing seems to be the only solution to find log entries if you don't fully know what you're looking for. For example: You want to see all entries containing a certain MESSAGE that gets enriched with additional information during the logging process: MESSAGE=host HOST has closed connection CONNECTION_ID This is a bit contentious, but at least I would like to see some grep functionality implemented directly in journalctl. I am late to the party, but I think it is obvious that the right way for this to be achieved, in a perfect world, is that this log entry be accompanied by a MESSAGE_ID, and HOST and CONNECTION_ID keys, and a catalog entry that combined with the keys, generates the above message so that grepping is entirely unnecessary. It is true that this perfect world is not just around the corner, or anything like that, but it is technically possible. I agree that grepping would be handy for me, right now, for just the reasons stated in the original message. I wonder if it would be reasonable for journalctl to supply the (additional) fields that are guaranteed to be associated with a MESSAGE_ID And what what happen when the entry is malformed, i.e. missing some fields? Would journald reject the message? I don't think this would be useful to anyone at all. Instead the readers of the message should gracefully adapt to missing fields. I think it would be wrong for journald to reject a message that does not supply all the declared fields. It would also be wrong for journalctl to crash when given the --catalog flag if the fields are missing. I don't know what it does right now, because it is not that easy a situation to engineer, AFAICT. I guess the best thing would be to supply a special catalog message indicating that an error had occurred when trying to construct a catalog message. Something that indicated the fields that were missing that caused the error would be nice. Just so long as that didn't turn into an infinite loop, somehow. If somebody knows what journalctl does do in this situation, please pass that information along. Re-reading the docs, I realize that the information is right there in plain sight. If the value is not defined, the variable name, minus the @ signs, is displayed. Nice and simple. But now, I wonder what happens for fields like _UDEV_DEVLINK, which can be set 0, 1, or many times. If unset, the variable name minus the @ signs is displayed. If set once, the value is substituted. If set twice? -- SNIP -- ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel - mulhern ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] RFC: filter and search journalctl
- Original Message - From: Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl To: Sebastian Schindler sebastian.schind...@travelping.com Cc: systemd-devel@lists.freedesktop.org Sent: Saturday, August 8, 2015 3:48:30 PM Subject: Re: [systemd-devel] RFC: filter and search journalctl On Fri, Aug 07, 2015 at 11:53:13AM +0200, Sebastian Schindler wrote: Grep-ing seems to be the only solution to find log entries if you don't fully know what you're looking for. For example: You want to see all entries containing a certain MESSAGE that gets enriched with additional information during the logging process: MESSAGE=host HOST has closed connection CONNECTION_ID This is a bit contentious, but at least I would like to see some grep functionality implemented directly in journalctl. I am late to the party, but I think it is obvious that the right way for this to be achieved, in a perfect world, is that this log entry be accompanied by a MESSAGE_ID, and HOST and CONNECTION_ID keys, and a catalog entry that combined with the keys, generates the above message so that grepping is entirely unnecessary. It is true that this perfect world is not just around the corner, or anything like that, but it is technically possible. I agree that grepping would be handy for me, right now, for just the reasons stated in the original message. I wonder if it would be reasonable for journalctl to supply the (additional) fields that are guaranteed to be associated with a MESSAGE_ID, and how this information might be registered. One way is to essentially derive this from an associated catalog entry, if any. Any fields that the catalog entry uses really ought to be supplied along w/ the MESSAGE_ID. This mapping is available to any human being, of course, by inspecting journal entries. But it also seems likely that there might be fields that should be guaranteed to accompany a MESSAGE_ID that should not be incorporated into a catalog message. I would be interested in the idea of, e.g., extending the format of the catalog file that an application distributes to allow an extra line that specifies guaranteed fields, or alternatively, to allow an additional file, dedicated to specifying this interface. This is a bit analogous to the interface file that is specified for foreign language bindings for a library. I'm also curious about a mechanism to distinguish those entries that are supplied specifically for a particular MESSAGE_ID from those that are, e.g., auto-generated by systemd or derived from some other sources. systemd has already taken the underscore for the unfakeable entries it provides. Is it reasonable to preface any MESSAGE_ID specific keys with the MESSAGE_ID, e.g., 9bb33380-fbfa-4d5b-88b5-6e6bb8a39124:KEY? Or perhaps a double underscore, e.g., __KEY would do the trick? -- SNIP -- Zbyszek ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel - mulhern ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] RFC: filter and search journalctl
- Original Message - From: Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl To: Anne Mulhern amulh...@redhat.com Cc: systemd-devel@lists.freedesktop.org, Sebastian Schindler sebastian.schind...@travelping.com Sent: Monday, August 17, 2015 10:45:11 AM Subject: Re: [systemd-devel] RFC: filter and search journalctl On Mon, Aug 17, 2015 at 10:24:22AM -0400, Anne Mulhern wrote: - Original Message - From: Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl To: Sebastian Schindler sebastian.schind...@travelping.com Cc: systemd-devel@lists.freedesktop.org Sent: Saturday, August 8, 2015 3:48:30 PM Subject: Re: [systemd-devel] RFC: filter and search journalctl On Fri, Aug 07, 2015 at 11:53:13AM +0200, Sebastian Schindler wrote: Grep-ing seems to be the only solution to find log entries if you don't fully know what you're looking for. For example: You want to see all entries containing a certain MESSAGE that gets enriched with additional information during the logging process: MESSAGE=host HOST has closed connection CONNECTION_ID This is a bit contentious, but at least I would like to see some grep functionality implemented directly in journalctl. I am late to the party, but I think it is obvious that the right way for this to be achieved, in a perfect world, is that this log entry be accompanied by a MESSAGE_ID, and HOST and CONNECTION_ID keys, and a catalog entry that combined with the keys, generates the above message so that grepping is entirely unnecessary. It is true that this perfect world is not just around the corner, or anything like that, but it is technically possible. I agree that grepping would be handy for me, right now, for just the reasons stated in the original message. I wonder if it would be reasonable for journalctl to supply the (additional) fields that are guaranteed to be associated with a MESSAGE_ID And what what happen when the entry is malformed, i.e. missing some fields? Would journald reject the message? I don't think this would be useful to anyone at all. Instead the readers of the message should gracefully adapt to missing fields. I think it would be wrong for journald to reject a message that does not supply all the declared fields. It would also be wrong for journalctl to crash when given the --catalog flag if the fields are missing. I don't know what it does right now, because it is not that easy a situation to engineer, AFAICT. I guess the best thing would be to supply a special catalog message indicating that an error had occurred when trying to construct a catalog message. Something that indicated the fields that were missing that caused the error would be nice. Just so long as that didn't turn into an infinite loop, somehow. If somebody knows what journalctl does do in this situation, please pass that information along. Other consumers of log entries should behave in whatever manner seems best to them if a declared field is missing. What I'm looking for here is the best way for an application which wants to use the journaling facilities provided to publish useful information about its log entry API. The advantage of publishing it in the manner I've suggested is that journalctl could be very helpful about telling consumers of the journal what keys they should expect to see. Something like: journalctl --list-keys MESSAGE_ID and maybe even a journal API for programmatic access to this information would be very nice. Of course, there are other ways for an application to publish its log entry API. But, it does seem odd for it to do this outside the structures that systemd has already set up, when it is an API for journal entries. Since this really is an API, with all the usual issues about versioning and so forth, it really is essential that the information be published somewhere, not laboriously extracted from a scan of the code by potential log entry consumers. ... Is it reasonable to preface any MESSAGE_ID specific keys with the MESSAGE_ID, e.g., 9bb33380-fbfa-4d5b-88b5-6e6bb8a39124:KEY? Or perhaps a double underscore, e.g., __KEY would do the trick? MESSAGE_ID is a contrace between the writers of the message and the readers of the message. The first say: messages with this ID mean ... and have have the fields ... . There is no need to mark the fields in any other way, except by documentation or custom. Zbyszek The reason this seems important to me is the problem of a shared namespace. These MESSAGE_ID UUIDs are globally registered, since there is a high enough probability that every UUID is different that they are, to all intents and purposes, unique. But the keys do not have this advantage. In this shared namespace, it would be easy enough for journald to steal a key that was already in use by another application. This would generate all the obvious and usual problems, most probably forcing
Re: [systemd-devel] Looking for experiences formalizing an API for journal messages
- Original Message - From: Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl To: Anne Mulhern amulh...@redhat.com Cc: systemd-devel@lists.freedesktop.org Sent: Thursday, July 30, 2015 10:01:54 AM Subject: Re: [systemd-devel] Looking for experiences formalizing an API for journal messages On Wed, Jul 29, 2015 at 03:02:26PM -0400, Anne Mulhern wrote: What I'm wondering about is the existence of some processes (not systemd), that have an agreement on a set of key-value pairs that they communicate with through the journal. There was work done on converting abrt to use the journal. We extended our set of metadata fields for coredumps which were already used internally by coredumpctl (see http://cgit.freedesktop.org/systemd/systemd/commit/?id=3f132692e3). I think this work is not finished yet, because of some issues that abrt would have to copy the coredump file (?), but abrt is becoming an external consumer. Thanks! That's a helpful datapoint. fail2ban has a systemd backend which uses the journal. It uses the python API for journal to add matches (the mechanism is general and the matches themselves are specified by filters). This is the same functionality that journalctl uses. I did take a look at fail2ban. I didn't study it in depth, but it looks like it processes journal entries into a different format and then does regular expression matching on the result. So it didn't really feel like a good example for what I had in mind. I've been exercising the systemd-python package a little and AFAICT it's a pretty straightforward mapping onto the journal C API, with a few extra bits from the journalctl front-end thrown in for convenience. HTH, Zbyszek - mulhern ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] in-kernel structured logging
Hi! This is really closely related to my previous question re. formalizing an API. In the design document (https://docs.google.com/document/pub?id=1IC9yOXj7j6cdLLxWEBAGRL6wl97tFxgjLUEHIX3MSTs) there is a statement about a future intention to support in-kernel structured logging. Are things like dev_printk() considered to have achieved that, or is something more planned for the future? Thanks again, - mulhern ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] in-kernel structured logging
- Original Message - From: Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl To: Anne Mulhern amulh...@redhat.com Cc: systemd-devel@lists.freedesktop.org Sent: Thursday, July 30, 2015 1:04:55 PM Subject: Re: [systemd-devel] in-kernel structured logging On Thu, Jul 30, 2015 at 12:16:03PM -0400, Anne Mulhern wrote: Hi! This is really closely related to my previous question re. formalizing an API. In the design document (https://docs.google.com/document/pub?id=1IC9yOXj7j6cdLLxWEBAGRL6wl97tFxgjLUEHIX3MSTs) there is a statement about a future intention to support in-kernel structured logging. Are things like dev_printk() considered to have achieved that, or is something more planned for the future? One thing is the basic support for structured logs, which was implemented, another is patching all the places which produce logs to attach useful metadata. This is only partially done. If you look at kernel logs from a boot, even basic things like the priority are set semi-randomly. So from the kernel the state could be described as POC. Where can the support for structured logging be found in the kernel? It seems tricky, given the kernel's constraints, to support arbitrary structured logging. I'm likely missing something, but as far as I can tell, dev_printk() is like printk(), except that it takes a few more parameters from which a few more fields about the device are automatically extracted and added to the buffer. Maybe dev_printk is one instance of what is considered the correct way to do structured logging in the kernel? From systemd side, we have the fields, but we don't do interesting things with them. One problem is that the kernel gives us tags like _KERNEL_DEVICE=+scsi:8:0:3:0 which are dynamic, and change between boots. For the logs to be useful, we would have to map those to some static identifiers. It would be nice to tell journalctl give me all logs about /dev/sda, but we're not there yet. Zbyszek - mulhern ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Looking for experiences formalizing an API for journal messages
- Original Message - From: Lennart Poettering lenn...@poettering.net To: Anne Mulhern amulh...@redhat.com Cc: systemd-devel@lists.freedesktop.org Sent: Thursday, July 23, 2015 9:21:44 AM Subject: Re: [systemd-devel] Looking for experiences formalizing an API for journal messages On Thu, 23.07.15 09:17, Anne Mulhern (amulh...@redhat.com) wrote: Hi! We all know that using the journald native API it is possible to enrich the log entry data w/ key/value pairs, although this facility is Linux only. The set of key/value pairs which a message may log to the journal can constitute an API with which a logging entity can communicate alerts to consumers of this information. Clearly this requires an agreement on an API between the logging entity and the consumer of the journal entries. Are there existing projects that have used this facility in a principled, coordinated way with some success or have there been any interesting failures along those same lines? systemd of courses uses its on its own, and we tried to document the fields we use in systemd.journal-fields(7), though it might be slightly incomplete. But yes, this is indeed API, and deserves complete documentation, like any C API, any D-Bus API or a any REST API would need too. Lennart -- Lennart Poettering, Red Hat What I'm wondering about is the existence of some processes (not systemd), that have an agreement on a set of key-value pairs that they communicate with through the journal. It seems like, even though there is a mechanism for adding additional key/value pairs, regular expression matching is still kind of the norm. I was hoping for an inspiring counter example that uses mostly or exclusively key/value pairs instead of expression matching, and maybe some hints as to best practices. - mulhern ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] Looking for experiences formalizing an API for journal messages
Hi! We all know that using the journald native API it is possible to enrich the log entry data w/ key/value pairs, although this facility is Linux only. The set of key/value pairs which a message may log to the journal can constitute an API with which a logging entity can communicate alerts to consumers of this information. Clearly this requires an agreement on an API between the logging entity and the consumer of the journal entries. Are there existing projects that have used this facility in a principled, coordinated way with some success or have there been any interesting failures along those same lines? Thanks for your help, - mulhern ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Confusing journal information - journal size
- Original Message - From: Colin Guthrie gm...@colin.guthr.ie To: systemd-devel@lists.freedesktop.org Sent: Monday, July 20, 2015 5:38:38 AM Subject: Re: [systemd-devel] Confusing journal information - journal size David Sommerseth wrote on 17/07/15 14:28: On 17/07/15 13:31, Mantas Mikulėnas wrote: On Fri, Jul 17, 2015 at 2:13 PM, David Sommerseth dav...@redhat.com mailto:dav...@redhat.com wrote: Hi, I'm looking through some journals now, and even though I've seen it a few times I haven't thought about it until now. systemd-journal[1151]: Runtime journal is using 8.0M (max allowed 4.0G, trying to leave 4.0G free of 63.7G available → current limit 4.0G). Could this line be cleaned up so you don't have to look up a man page to try to figure out what this really means? Here's my uneducated guess and confusion of this line: * Runtime journal is using 8.0M - Okay, so currently the journal uses 8MB of disk-space. No problem. * max allowed 4.0G - Okay, so the journal should not grow beyond 4GB, makes sense. No problem. * trying to leave 4.0G free of 63.7G available - Uhm, what!? So it will grow until there is 4GB left on the filesystem? Not so okay. It chooses the /smallest/ limit, not largest. (Common sense...) For example, if you had only 5 GB space available, the journal would not grow beyond 1 GB. * current limit 4.0G - Ehh ... okay ... so make up your mind, please! So will the journal grow until 4GB or 59.7GB. This *is* it making up its mind: min(limit 1, limit 2) → resulting limit But then I looked into /var/log/journal ... # du --si -s /var/log/journal/ 4.3G /var/log/journal/ I do see that both system,journal and user-UID.journal are both 8.4MB, and from that I can guess what the log entry tried to tell me with Runtime journal ... but how is /that/ information useful for me, from a sys-admin point of view? Runtime here means /run, as opposed to persistent in /var. They have separately configurable limits, since /run is in RAM and /var is usually on disk. (Though, I'm not entirely sure what purpose the runtime journal even serves, when /var is available.) Fair enough. But you are missing my point. How this information is presented do require some detail knowledge of the journal. Don't think like a developer who have poked at the journal code. Think like a sys-admin who looks through the logs looking for issues. Then you want to have the answer straight in your face, not needing to go elsewhere to read about these things. In fact most admins will probably have forgotten what they were going to look for when they move their eyes of the log data. If it is considered important information, fine. But present it in a far more understandable way for those who just uses the journal. Right now, I'm not surprised if most sys-admins read that line as useless gibberish - Yeah, yeah, journal will waste some space on my drive. Yeah, I can't disagree with David. Not sure how best to tidy it up, but some rework would definitely be nice. Col -- Colin Guthrie gmane(at)colin.guthr.ie http://colin.guthr.ie/ Day Job: Tribalogic Limited http://www.tribalogic.net/ Open Source: Mageia Contributor http://www.mageia.org/ PulseAudio Hacker http://www.pulseaudio.org/ Trac Hacker http://trac.edgewall.org/ ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel After seeing the explanation, the best complete and correct (AFAICT) formulation I could come up with was, Runtime journal is using 8.0M (max allowed = min(4.0G, S s.t. total memory(63.7 G) - S = 4.0 G (59.7 G), available memory (16.2 G)) = 4.0G) which is compelled to use math speak for clarity and succinctness. Dunno how happy most sys-admins would be with that. - mulhern ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Confusing journal information - journal size
- Original Message - From: David Sommerseth dav...@redhat.com To: Anne Mulhern amulh...@redhat.com Cc: systemd-devel@lists.freedesktop.org Sent: Monday, July 20, 2015 11:29:22 AM Subject: Re: [systemd-devel] Confusing journal information - journal size On 20/07/15 15:31, Anne Mulhern wrote: - Original Message - From: Colin Guthrie gm...@colin.guthr.ie To: systemd-devel@lists.freedesktop.org Sent: Monday, July 20, 2015 5:38:38 AM Subject: Re: [systemd-devel] Confusing journal information - journal size David Sommerseth wrote on 17/07/15 14:28: On 17/07/15 13:31, Mantas Mikulėnas wrote: On Fri, Jul 17, 2015 at 2:13 PM, David Sommerseth dav...@redhat.com mailto:dav...@redhat.com wrote: Hi, I'm looking through some journals now, and even though I've seen it a few times I haven't thought about it until now. systemd-journal[1151]: Runtime journal is using 8.0M (max allowed 4.0G, trying to leave 4.0G free of 63.7G available → current limit 4.0G). Could this line be cleaned up so you don't have to look up a man page to try to figure out what this really means? Here's my uneducated guess and confusion of this line: * Runtime journal is using 8.0M - Okay, so currently the journal uses 8MB of disk-space. No problem. * max allowed 4.0G - Okay, so the journal should not grow beyond 4GB, makes sense. No problem. * trying to leave 4.0G free of 63.7G available - Uhm, what!? So it will grow until there is 4GB left on the filesystem? Not so okay. It chooses the /smallest/ limit, not largest. (Common sense...) For example, if you had only 5 GB space available, the journal would not grow beyond 1 GB. * current limit 4.0G - Ehh ... okay ... so make up your mind, please! So will the journal grow until 4GB or 59.7GB. This *is* it making up its mind: min(limit 1, limit 2) → resulting limit But then I looked into /var/log/journal ... # du --si -s /var/log/journal/ 4.3G /var/log/journal/ I do see that both system,journal and user-UID.journal are both 8.4MB, and from that I can guess what the log entry tried to tell me with Runtime journal ... but how is /that/ information useful for me, from a sys-admin point of view? Runtime here means /run, as opposed to persistent in /var. They have separately configurable limits, since /run is in RAM and /var is usually on disk. (Though, I'm not entirely sure what purpose the runtime journal even serves, when /var is available.) Fair enough. But you are missing my point. How this information is presented do require some detail knowledge of the journal. Don't think like a developer who have poked at the journal code. Think like a sys-admin who looks through the logs looking for issues. Then you want to have the answer straight in your face, not needing to go elsewhere to read about these things. In fact most admins will probably have forgotten what they were going to look for when they move their eyes of the log data. If it is considered important information, fine. But present it in a far more understandable way for those who just uses the journal. Right now, I'm not surprised if most sys-admins read that line as useless gibberish - Yeah, yeah, journal will waste some space on my drive. Yeah, I can't disagree with David. Not sure how best to tidy it up, but some rework would definitely be nice. Col After seeing the explanation, the best complete and correct (AFAICT) formulation I could come up with was, Runtime journal is using 8.0M (max allowed = min(4.0G, S s.t. total memory(63.7 G) - S = 4.0 G (59.7 G), available memory (16.2 G)) = 4.0G) which is compelled to use math speak for clarity and succinctness. Dunno how happy most sys-admins would be with that. - mulhern But is all that information really needed? If I try to see this from a sys-admin point of view there are two numbers which are important to me: 1) Current state 2) Final journal limit size. From how I see it, how the journal code ends up with a certain number is only useful when you're developing/debugging the journal. Remember: Less is more. -- kind regards, David Sommerseth That seems reasonable to me. And Runtime journal is using 8.0M (max allowed = 4.0G) is telling it like it is (for this system). The additional info really does seem like its appropriate situation is in debugging output. Also, now that I look at the source it seems like my formulation was wrong, journal is claiming that current limit is s-cached_available_space + sum, which isn't really the minimum of any of the other values mentioned. So something more is going on. - mulhern
[systemd-devel] Fwd: kernel logging_journald_journalctl question
Advice appreciated. Thanks, - mulhern - Forwarded Message - From: Anne Mulhern amulh...@redhat.com To: OS Development List os-devel-l...@redhat.com Sent: Thursday, July 9, 2015 4:46:13 PM Subject: kernel logging_journald_journalctl question Hi! My ultimate goal is that the following example line: journalctl _WWN=wwn-0x5002e100 should be meaningful, i.e., * yield all log messages pertaining to a device w/ that particular wwn * the _WWN value should be guaranteed with the same certainty as journalctl's other unfakeable values with underscores, e.g., _UID. One subgoal is, therefore, to make sure that _correct_ _WWN information gets into the journal when that information is available. In principle, journal messages about devices that possess WWN's might originate in a variety of places. This makes things complicated, since it is possible that the appropriate way of discovering the WWN may vary with the origin of the message. (For example, lsscsi discovers the WWN of the current device by searching /dev/disk/by-id/.) However, I'm focusing on messages that originate in the kernel via dev_printk() in the SCSI mid-layer. I infer that journald reads them from the kernel log buffer in the dev_kmsg_record() method. I also know that dev_printk guarantees to print two key/value pairs, SUBSYSTEM and DEVICE. What is the best way to make information about the WWN available to journald? Can it reliably extract this information from the DEVICE field in the dev_kmsg_record() method? Is there a process that I don't know about that picks up messages from the kernel and augments them with extra metadata before they end up in /dev/kmsg that can be modified? Would dev_printk() itself have to be extended to include WWN information as another field in addition to SUBSYSTEM and DEVICE? Must I fall back on making sure that individual calls to dev_printk() include some essential information that can be parsed back out by journald? It would also be nice to take into consideration that messages regarding devices could be logged from other sources, like the journald API, and it would be nice to reliably store the WWN information from those messages in the journal, as well. Thanks in advance for any help, - mulhern ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
