This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 78e20d5cbd577878ba2451e13db516e07a9629cb
Author: Benoit Tellier <btell...@linagora.com>
AuthorDate: Wed Feb 12 14:52:40 2020 +0700
JAMES-3048 Distributed admin procedured: Mail processing
Context about mail processing components, details about when and how to
reprocess emails in mail repositories.
Includes diagnostic via logs and metrics.
---
.../server/manage-guice-distributed-james.md | 65 ++++++++++++++++++++--
1 file changed, 60 insertions(+), 5 deletions(-)
diff --git a/src/site/markdown/server/manage-guice-distributed-james.md
b/src/site/markdown/server/manage-guice-distributed-james.md
index 8a2532a..57f0698 100644
--- a/src/site/markdown/server/manage-guice-distributed-james.md
+++ b/src/site/markdown/server/manage-guice-distributed-james.md
@@ -17,6 +17,7 @@ advanced users.
- [Overall architecture](#overall-architecture)
- [Basic Monitoring](#basic-monitoring)
+ - [Mail Processing](#mail-processing)
## Overall architecture
@@ -84,10 +85,64 @@ Here are the available checks alongside the insight they
offer:
- **Cassandra backend**: Cassandra storage. Ensure queries can be executed on
the connection James uses.
- **ElasticSearch Backend**: ElasticSearch storage. Triggers an ElasticSearch
health request on indices James uses.
- **RabbitMQ backend**: RabbitMQ messaging. Verifies an open connection and
an open channel are well available.
- - **Guice application lifecycle**: Ensures James Guice successfully started,
and is up. Logs should contain explanations if
- James did not start well.
- - **MessageFastViewProjection**: Follows MessageFastViewProjection cache miss
rates and warns if it is below 10%. If this
- projection is missing, this results in performance issues for JMAP
GetMessages list requests. WebAdmin offers a
+ - **Guice application lifecycle**: Ensures James Guice successfully started,
and is up. Logs should contain
+ explanations if James did not start well.
+ - **MessageFastViewProjection**: Follows MessageFastViewProjection cache miss
rates and warns if it is below 10%. If
+ this projection is missing, this results in performance issues for JMAP
GetMessages list requests. WebAdmin offers a
[global](manage-webadmin.html#recomputing-global-jmap-fast-message-view-projection)
and
[per
user](manage-webadmin.html#recomputing-user-jmap-fast-message-view-projection)
projection re-computation. Note that
- as computation is asynchronous, this projection can be slightly out of sync
on a normally behaving server.
\ No newline at end of file
+ as computation is asynchronous, this projection can be slightly out of sync
on a normally behaving server.
+
+## Mail Processing
+
+Mail processing allows to take asynchronously business decisions on received
emails.
+
+Here are its components:
+
+ - The `spooler` takes mail out of the mailQueue and executes mail processing
within the `mailet container`.
+ - The `mailet container` synchronously executes the user defined logic. This
'logic' is written through the use of
+ `mailet`, `matcher` and `processor`.
+ - A `mailet` represents an action: mail modification, envelop modification, a
side effect, or stop processing.
+ - A `matcher` represents a condition to execute a mailet.
+ - A `processor` is a flow of pair of `matcher` and `mailet` executed
sequentially. The `ToProcessor` mailet is a `goto`
+ instruction to start executing another `processor`
+ - A `mail repository` allows storage of a mail as part of its processing.
Standard configuration relies on the
+ following mail repository:
+ - `cassandra://var/mail/error/` : unexpected errors that occurred during
mail processing. Emails impacted by
+ performance related exceptions, or logical bug within James code are
typically stored here. These mails could be
+ reprocessed once the cause of the error is fixed. The `Mail.error` field
can help diagnose the issue. Correlation
+ with logs can be achieved via the use of the `Mail.name` field.
+ - `cassandra://var/mail/address-error/` : mail addressed to a
non-existing recipient of a handled local domain.
+ These mails could be reprocessed once the user is created, for instance.
+ - `cassandra://var/mail/relay-denied/` : mail for whom relay was denied:
missing authentication can, for instance,
+ be a cause. In addition to prevent disasters upon miss configuration, an
email review of this mail repository can
+ help refine a host spammer blacklist.
+ - `cassandra://var/mail/rrt-error/` : runtime error upon Recipient
Rewritting occurred. This is typically due to a
+ loop. We recommend verifying user mappings via [User Mappings webadmin
API](manage-webadmin.html#user-mappings)
+ then once identified break the loop by removing some Recipient Rewrite
Table entry via the
+ [Delete Alias](manage-webadmin.html#removing-an-alias-of-an-user),
+ [Delete Group member](manage-webadmin.html#removing-a-group-member),
+ [Delete
forward](manage-webadmin.html#removing-a-destination-of-a-forward),
+ [Delete Address mapping](manage-webadmin.html#remove-an-address-mapping),
+ [Delete Domain mapping](manage-webadmin.html#removing-a-domain-mapping)
or
+ [Delete Regex mapping](manage-webadmin.html#removing-a-regex-mapping)
APIs (as needed). The `Mail.error` field can
+ help diagnose the issue as well. Then once the root cause has been
addressed, the mail can be reprocessed.
+
+Read [this](config-mailetcontainer.html) to discover mail processing
configuration, including error management.
+
+Currently, an administrator can monitor mail processing failure through
`ERROR` log review. We also recommend watching
+in Kibana INFO logs using the `org.apache.james.transport.mailets.ToProcessor`
value as their `logger`. Metrics about
+mail repository size, and the corresponding Grafana boards are yet to be
contributed.
+
+WebAdmin exposes all utilities for
+[reprocessing all mails in a mail
repository](manage-webadmin.html#reprocessing-mails-from-a-mail-repository)
+or
+[reprocessing a single mail in a mail
repository](manage-webadmin.html#reprocessing-a-specific-mail-from-a-mail-repository).
+
+Also, one can decide to
+[delete all the mails of a mail
repository](manage-webadmin.html#removing-all-mails-from-a-mail-repository)
+or [delete a single mail of a mail
repository](manage-webadmin.html#removing-a-mail-from-a-mail-repository).
+
+Performance of mail processing can be monitored via the
+[mailet grafana
board](https://github.com/apache/james-project/blob/master/grafana-reporting/MAILET-1490071694187-dashboard.json)
+and [matcher grafana
board](https://github.com/apache/james-project/blob/master/grafana-reporting/MATCHER-1490071813409-dashboard.json).
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
