Re: [systemd-devel] Online backup API for systemd-journal?
On Mo, 04.09.23 16:35, Etienne Doms (etienne.d...@gmail.com) wrote: > Hi, > > I have some embedded systems in the wild, not connected to anything, > on which you can push a button "something went wrong, create a dump". > Then later I can fetch the said dump and inspect it. > > I'd like to include the whole journal, for the current boot, in a > binary format so that I can later do "journalctl --file > path/to/journal-dump.bin" from another machine. I understand that > internally everything is stored in /var/log/journal/, but > I guess that I cannot blindly tar/cp the .journal files, since this > would be racy. That should actually work fine. journald has no locking around journal files: the server that writes to the files and the client that reads them are not synchronized. The client is supposed to handle incomplete writes by simply suppressing display of the trailing, incomplete entries. This is a common code path, that is quite well tested these days. Hence, I should actually be fine to just copy the journal files as they are being written, the tools on the other side will possibly then see a file with records currently "in flight" that are referenced at some places but not others, but that should be totally OK, the tools should handle this, and this i no different from their local access. > So, is there an API to safely dump a big ".journal" file containing a > snapshot of "journalctl -b"? I could not find anything in the > documentation, sorry in advance if I missed something obvious. You can use "-o export" to dumb the files in an "export" format. But this is just about returning the data in a different format, it does not give you any synchronization guarantess since journalctl started that way will just read the data from the journal files unsynchronized as everyeone else too. Lennart -- Lennart Poettering, Berlin
Re: [systemd-devel] Online backup API for systemd-journal?
Oh, I knew I was missing something with the "-o export", I wasn't aware of systemd-journal-remote. Thanks! Le lun. 4 sept. 2023 à 18:54, Mantas Mikulėnas a écrit : > > On Mon, Sep 4, 2023 at 5:35 PM Etienne Doms wrote: >> >> Hi, >> >> I have some embedded systems in the wild, not connected to anything, >> on which you can push a button "something went wrong, create a dump". >> Then later I can fetch the said dump and inspect it. >> >> I'd like to include the whole journal, for the current boot, in a >> binary format so that I can later do "journalctl --file >> path/to/journal-dump.bin" from another machine. I understand that >> internally everything is stored in /var/log/journal/, but >> I guess that I cannot blindly tar/cp the .journal files, since this >> would be racy. >> >> So, is there an API to safely dump a big ".journal" file containing a >> snapshot of "journalctl -b"? I could not find anything in the >> documentation, sorry in advance if I missed something obvious. > > > Run `journalctl --rotate` (or send a SIGUSR2). All "rotated" .journal files > (containing an '@' in their name) are offline and can be copied. > >> For now I just dump it with "-o json" which is fine, but then I cannot >> feed another journalctl with the given json, and need to do manual >> filtering. > > > If you dump with `-o export` instead (or convert the JSON to the export > format), you can later feed the dump into systemd-journal-remote(8) (which is > somewhere in /lib/systemd) to import it back into a .journal file. > > -- > Mantas Mikulėnas
Re: [systemd-devel] Online backup API for systemd-journal?
On Mon, Sep 4, 2023 at 5:35 PM Etienne Doms wrote: > Hi, > > I have some embedded systems in the wild, not connected to anything, > on which you can push a button "something went wrong, create a dump". > Then later I can fetch the said dump and inspect it. > > I'd like to include the whole journal, for the current boot, in a > binary format so that I can later do "journalctl --file > path/to/journal-dump.bin" from another machine. I understand that > internally everything is stored in /var/log/journal/, but > I guess that I cannot blindly tar/cp the .journal files, since this > would be racy. > > So, is there an API to safely dump a big ".journal" file containing a > snapshot of "journalctl -b"? I could not find anything in the > documentation, sorry in advance if I missed something obvious. > Run `journalctl --rotate` (or send a SIGUSR2). All "rotated" .journal files (containing an '@' in their name) are offline and can be copied. For now I just dump it with "-o json" which is fine, but then I cannot > feed another journalctl with the given json, and need to do manual > filtering. If you dump with `-o export` instead (or convert the JSON to the export format), you can later feed the dump into systemd-journal-remote(8) (which is somewhere in /lib/systemd) to import it back into a .journal file. -- Mantas Mikulėnas
[systemd-devel] Online backup API for systemd-journal?
Hi, I have some embedded systems in the wild, not connected to anything, on which you can push a button "something went wrong, create a dump". Then later I can fetch the said dump and inspect it. I'd like to include the whole journal, for the current boot, in a binary format so that I can later do "journalctl --file path/to/journal-dump.bin" from another machine. I understand that internally everything is stored in /var/log/journal/, but I guess that I cannot blindly tar/cp the .journal files, since this would be racy. So, is there an API to safely dump a big ".journal" file containing a snapshot of "journalctl -b"? I could not find anything in the documentation, sorry in advance if I missed something obvious. For now I just dump it with "-o json" which is fine, but then I cannot feed another journalctl with the given json, and need to do manual filtering. Thanks, Etienne
