Le 03/12/2018 à 19:25, David Fetter a écrit : > On Mon, Dec 03, 2018 at 07:18:31PM +0100, Gilles Darold wrote: >> Le 30/11/2018 à 19:53, David Fetter a écrit : >>> This makes it much simpler for computers to use the logs while not >>> making it excessively difficult for humans to use them. >> I'm also not very comfortable with this change. For a pgBadger point >> of view I don't know an existing library to parse csv log in >> multi-process mode. The only way I know is to split the file or load >> chunks in memory which is not really recommended with huge log >> files. I am not saying that this is not possible to have a Perl CSV >> library allowing multi-process but this require some additional days >> of work. > How about other structured log formats? Would one for JSON work > better?
pgBadger is able to parse jsonlog (https://github.com/michaelpq/pg_plugins/tree/master/jsonlog) output in multi-process mode because it do not use an external library to parse the json. In this minimalist implementation I assume that each log line is a full json object. Honestly I do not have enough feedback on this format to be sure that my basic jsonlog parser is enough, maybe it is not. What is really annoying for a log parser is the preservation of newline character in queries, if the format outputs each atomic log messages on a single line each, this is ideal. > >> Also at this time I have not received any feature request for that, >> maybe the use of csvlog format is not so common. > Part of the reason it's uncommon is that it's not the default we ship > with, and defaults influence this very much. I agree. I also think that syslog or stderr are more human readable than csvlog or jsonlog, which have probably contributed to their large adoption too. -- Gilles Darold http://www.darold.net/