I think a move to structured logging will be very helpful. As @eiri points out,
at least in my experience it makes one think more deeply about useful log ...
_content_ I'll call it 😄
FWIW I have tended to think of the "message" as the human readable bit that
will show up by default in tools like Kibana or Splunk. In this thinking should
therefore be a useful one line to scan. So for the compaction example, I would
likely choose to output the message of "Compaction started for shards/....."
rather than just "Compaction started.". I think this is my only main insight
that I haven't seen in the awesome discussion on this ticket so far :)
While I think @eiri is right that message is "just another field", most
solutions appear to output a `message` field by default in the UI and have
disclosure arrows to view other things, so I do think that these messages need
to be useful for a human reading the logs.
For the other fields, I've tended to think about what would be useful from a
searching perspective. So "action: compaction_start" might be a useful one that
these examples miss out. Broadly: what should I include that means that a
search doesn't have to include free-text searches of the message field. In
practice this usually means duplicated information between the message and the
fields.
For the API specifics:
- I side with @eiri that a single "bin" for the extra fields in the JSON is
easier, it's not obvious to me whether there's a natural mapping. For the
meta-info of line etc. perhaps the macro/parse transform needs to validate no
name clashes once its validated the log level? Problem here is that you need
good test code coverage to make sure this doesn't explode (could a transform do
this check at compile time?).
- I like something like:
couch_log:error(FormatString, FormatArgs, ExtraFields)
which ends up like:
couch_log:error("Compaction started for ~p", [{shard, "shards/..."}],
[{user, "foo"}, {shard, "shards/..."}])
Note the duplication of shard info, but I think this makes reading the logs
and searching them simpler. Arguably you could have named bit in the string
formatting that could read fields from `ExtraFields` to avoid having to have
the `FormatArgs` argument. I don't know if Erlang's formatting supports it
though.
- I have always liked the "log4X" pattern where what to log is separate from
the format of the log, and they are configured separately. At a scan, the
Erlang 21 logger @kocolosk links to follows this pattern; I've always liked
python's `logger` class too, though it's a bit long in the tooth.
[ Full content available at: https://github.com/apache/couchdb/issues/1373 ]
This message was relayed via gitbox.apache.org for [email protected]
