mikerhodes commented on issue #1373: Structured logging URL: https://github.com/apache/couchdb/issues/1373#issuecomment-417602224 I think a move to structured logging will be very helpful. As @eiri points out, at least in my experience it makes one think more deeply about useful log ... _content_ I'll call it 😄 FWIW I have tended to think of the "message" as the human readable bit that will show up by default in tools like Kibana or Splunk. In this thinking should therefore be a useful one line to scan. So for the compaction example, I would likely choose to output the message of "Compaction started for shards/....." rather than just "Compaction started.". I think this is my only main insight that I haven't seen in the awesome discussion on this ticket so far :) While I think @eiri is right that message is "just another field", most solutions appear to output a `message` field by default in the UI and have disclosure arrows to view other things, so I do think that these messages need to be useful for a human reading the logs. For the other fields, I've tended to think about what would be useful from a searching perspective. So "action: compaction_start" might be a useful one that these examples miss out. Broadly: what should I include that means that a search doesn't have to include free-text searches of the message field. In practice this usually means duplicated information between the message and the fields. For the API specifics: - I side with @eiri that a single "bin" for the extra fields in the JSON is easier, it's not obvious to me whether there's a natural mapping. For the meta-info of line etc. perhaps the macro/parse transform needs to validate no name clashes once its validated the log level? Problem here is that you need good test code coverage to make sure this doesn't explode (could a transform do this check at compile time?). - I like something like: couch_log:error(FormatString, FormatArgs, ExtraFields) which ends up like: couch_log:error("Compaction started for ~p", [{shard, "shards/..."}], [{user, "foo"}, {shard, "shards/..."}]) Note the duplication of shard info, but I think this makes reading the logs and searching them simpler. Arguably you could have named bit in the string formatting that could read fields from `ExtraFields` to avoid having to have the `FormatArgs` argument. I don't know if Erlang's formatting supports it though. - I have always liked the "log4X" pattern where what to log is separate from the format of the log, and they are configured separately. At a scan, the Erlang 21 logger @kocolosk links to follows this pattern; I've always liked python's `logger` class too, though it's a bit long in the tooth.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
