On Sat, Mar 14, 2015 at 11:07 AM, Zbigniew Jędrzejewski-Szmek <zbys...@in.waw.pl> wrote: > On Sat, Mar 14, 2015 at 10:59:42AM -0400, Chris Morgan wrote: >> On Fri, Mar 13, 2015 at 11:03 PM, Zbigniew Jędrzejewski-Szmek >> <zbys...@in.waw.pl> wrote: >> > On Fri, Mar 13, 2015 at 06:35:38PM -0400, Chris Morgan wrote: >> >> Hello. >> >> >> >> I posted this, >> >> http://lists.freedesktop.org/archives/systemd-devel/2013-July/011926.html, >> >> some time ago about tiered logging for embedded systems. >> >> >> >> The goal is to guarantee that the flash memory will last the duration >> >> of the product by carefully controlling who writes to it. >> >> >> >> I'm back looking at the issue and wanted to re-open the discussion. >> >> >> >> One approach that came up would be to set "Storage=volatile", a limit >> >> of say "10MB" for the journal size, and then write a separate program >> >> that would filter out the journal entries and write them to a file on >> >> a physical disk. >> > You should be able to do something like this right now with journal-remote: >> > journalctl --directory /run/log/<bootid> -o export | \ >> > systemd-journal-remote -o /var/log/<bootid>/ - >> > >> >> Thats pretty cool. I didn't realize systemd-journal-remote existed. >> Using journalctl would mean we have the ability to add filters to the >> output stream. > Yes. > >> > This is a bit too hacky to put into production, and it would be better >> > to have a single binary which does this. But all the parts are there: >> > >> > - following the journal and filtering, >> > - opening of specific directory as input, >> > - saving of state (i.e. the last cursor written), >> > - writing to a directory and rotating files the same as journald does. >> > >> > So adding a new binary (or extending one of the existing ones) >> > supporting your use case would be a matter of hooking stuff together. >> > >> >> The filtering portion is required as we are using the >> >> journal to persist some important information that we'd like to log. >> >> We'd also like to preserve high priority messages but don't mind if we >> >> lose low priority ones across reboots. >> >> >> >> An upside of the external program is that we can filter on both high >> >> priority messages as well as those with specific "FIELD=value" >> >> entries. Downside is a custom format file and can't use journalctl to >> >> search through it, no log rotation, no size limits etc. >> >> >> >> At the time there was some thought of putting this into journald >> >> itself. I'm wondering how that would fit given the desire to use more >> >> complicated matching to decide which entries were put into the >> >> persisted journal. >> > >> > Adding filtering and splitting functionality to journald is another >> > story of course. It probably would work better: more efficent, and >> > journal entries would not be duplicated at all. >> > >> >> The piping example you gave does seem to be a bit more heavyweight >> than a process that was using the sd_journal_xxx() calls to interact >> with the journal. >> >> It doesn't look like systemd-journal-remote supports size limits or >> rotation, the man page doesn't have any options for that. > It does. They are not configurable, so it just uses the defaults. > I guess that this should be added. >
Ahh. Let me take a look at adding those as a way to figure out how to test systemd in a container and get started developing. >> Would it be much less efficient to make yet another program that used >> sd_journal_xxx() calls, and whatever functionality >> systemd-journal-remote used to write to a new journal vs. trying to >> extend journald to do this internally? I'd rather not try to force >> things into journald proper that don't belong there if the separate >> application approach is nearly as performant and much cleaner. > One thing which implementing this in journald would give over implementing > is in something separate: journald could be configured to split messages > between /run, or /var. journalctl has the ability to transparently merge > events from multiple directories, so this split would be invisible to > users. > > I don't see how this could be done so nicely with a separate tool. > Yeah. Journalctl could blend the two together. I guess that implies that the entries put into the persisted journal would not make it into the non-persisted journal, eg. there wouldn't be duplicate entries. >> Or possibly enhance systemd-journal-remote to also support following >> the journal in addition to its current behavior of taking input from >> stdin? > Yeah, that seems like a good idea. > >> >> If it would fit inside of journald I'd be willing to implement it but >> >> we would need to come up with a way to configure the filtering, where >> >> the files are persisted etc. The filtering is a new requirement since >> >> the last time this was discussed. >> > I think that supporting a set of similar filters to journalctl would >> > be a good start. Options which limit the number of messages or >> > filter based on time would not make sense for journald, but most >> > other probably would. >> > >> >> Not following what you are referring to when you say "supporting a set >> of similar filters to journalctl would be a good start". > _FIELD=value, --priority, --unit, --user-unit, --dmesg, --identifier, --user, > --system. > > Zbyszek Ahh. Thought it supported a bit of that already. Chris _______________________________________________ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel