Benoit Tellier created JAMES-3399:
-------------------------------------
Summary: Document structured logging with FluentBit
Key: JAMES-3399
URL: https://issues.apache.org/jira/browse/JAMES-3399
Project: James Server
Issue Type: Improvement
Components: Documentation, guice
Affects Versions: master
Reporter: Benoit Tellier
== Context
As any application, James outputs logs.
The team uses the SLF4J logging facade, along with the logback-classic log
producer library.
The team had been setting up contextual logging: the context of the execution
is recorded through the use of the
MDC (Mapped Diagnostic Context) on the behalf of the developper. Once the
context set up, all logging action performed in the scope will implicilty
include the MDC context.
Contextual logging is beneficial as it enable dispaying efficiently request
parameters, tracking users, request ids and so on. It greatly enhence the
debugging experience.
Historically, we relyed on a custom logback appender to output logs directly to
ElasticSearch, in a structured fashion, that could be directly accessed via
Kibana.
The historical stack had been proven brittle:
- James needs to be pushing logs, eventual consummiung some of its resources
- Being push based, upon bolks or temporary ElasticSearch failures, some logs
are lost.
This conversation was marked as resolved by chibenwa
- Logs are duplicated, both to the standard output and ElasticSearch
- Changes to the logging stack requires a James restart, that could cause
inconsistency and harm availability.
This conversation was marked as resolved by chibenwa
== Desirable propoerties of the logging stack
We see the following properties as desirable:
- Having structured logging
- Being pull based
- Uncorrelating the logging stack from the James configuration
== Definition of Done
Document how to rely on FluentD to collect James logs. The James logging
configuration can be simplified to log all logs to the standard output.
Document how to emmit logs in a JSON format, in order for fluentD and Kibana to
allow preserving the structured logging experience.
== Consequences
We expect our logging stack to be more resilient, while keeping structured
logging.
https://github.com/linagora/logback-elasticsearch-appender project can be
deprecated - if desired.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
