This is an automated email from the ASF dual-hosted git repository. btellier pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/james-project.git
commit cccbdd788e29c9dc796a56591ec40f21ed021788 Author: Benoit Tellier <[email protected]> AuthorDate: Tue May 5 16:45:38 2020 +0700 [ADR] Write a standalone EventBus ADR to centralize and detail related knowledge --- ...oving-configured-additional-mailboxListeners.md | 3 ++ .../0027-eventBus-error-handling-upon-dispatch.md | 2 + .../0035-distributed-listeners-configuration.md | 4 +- src/adr/0036-eventbus.md | 63 ++++++++++++++++++++++ src/adr/0037-distributed-eventbus.md | 46 ++++++++++++++++ 5 files changed, 117 insertions(+), 1 deletion(-) diff --git a/src/adr/0026-removing-configured-additional-mailboxListeners.md b/src/adr/0026-removing-configured-additional-mailboxListeners.md index d1c8ceb..f81c59b 100644 --- a/src/adr/0026-removing-configured-additional-mailboxListeners.md +++ b/src/adr/0026-removing-configured-additional-mailboxListeners.md @@ -18,6 +18,9 @@ per listener. The distributed James server then declares a queue upon start for each one of these user registered listeners, that it binds to the main event exchange. +More information about this component, and its distributed, RabbitMQ based implementation, can be found in +[ADR 0036](0036-eventbus.md). + If the user unconfigures the listener, the queue and the binding are still present but not consumed. This results in unbounded queue growth eventually causing RabbitMQ resource exhaustion and failure. diff --git a/src/adr/0027-eventBus-error-handling-upon-dispatch.md b/src/adr/0027-eventBus-error-handling-upon-dispatch.md index 60e5cba..053b713 100644 --- a/src/adr/0027-eventBus-error-handling-upon-dispatch.md +++ b/src/adr/0027-eventBus-error-handling-upon-dispatch.md @@ -19,6 +19,8 @@ order to retry their processing via webAdmin APIs. However no such capabilities are supported upon dispatching the event on the eventbus. A failed dispatch will result in message loss. +More information about this component can be found in [ADR 0036](0036-eventbus.md). + ## Decision Upon dispatch failure, the eventBus should save events in dead letter using a dedicated group. diff --git a/src/adr/0035-distributed-listeners-configuration.md b/src/adr/0035-distributed-listeners-configuration.md index 626dfbb..78ada23 100644 --- a/src/adr/0035-distributed-listeners-configuration.md +++ b/src/adr/0035-distributed-listeners-configuration.md @@ -19,6 +19,8 @@ Mailbox listeners can be registered to be triggered every time an event is gener mailbox. They are being executed in a distributed fashion following the workqueue messaging pattern. The "group" is an attribute of the mailbox listener identifying to which work queue they belong. +More information about this component can be found in [ADR 0036](0036-eventbus.md). + Currently, mailbox listeners are determined by the guice bindings of the server and additional mailbox listeners defined via configuration files. @@ -76,7 +78,7 @@ A WebAdmin endpoint will allow: A broadcast on the event bus will be attempted to propagate topology changes, by the mean of a common registrationKey to all nodes, a "TopologyChanged" event, and a mailbox listener starting the MailboxListeners on local node upon -topology changes. +topology changes. `registrationKey` concept is explained in [ADR 0036](0036-eventbus.md). If a listener is added but is not in the classpath, an ERROR log is emitted. This can happen during a rolling upgrade, which defines a new guice binding for a new mailbox listener. Events will still be emitted (and consumed by other James diff --git a/src/adr/0036-eventbus.md b/src/adr/0036-eventbus.md new file mode 100644 index 0000000..e16f782 --- /dev/null +++ b/src/adr/0036-eventbus.md @@ -0,0 +1,63 @@ +# 36. Event bus + +Date: 2020-05-05 + +## Status + +Accepted (lazy consensus) + +## Context + +Many features rely on behaviors triggered by interactions with the mailbox API main interfaces (`RightManager`, +`MailboxManager`, `MessageManager`, `MessageIdManager`). We need to provide a convenient extension mechanism for +organizing the execution of these behaviours, provide retries and advanced error handling. + +Also, protocols enable notifications upon mailbox modifications. This is for instance the case for `RFC-2177 IMAP IDLE`, +leveraged for `RFC-3501 IMAP unsolicitated notifications` when selecting a Mailbox, as well as maintaining the +`IMAP Message Sequence Number <-> Unique IDentifier` MSN <-> UID mapping. Changes happening for a specific entity +(mailbox) need to be propagated to the relevant listeners. + +## Decision + +James mailbox component, a core component of James handling the storage of mails and mailboxes, should use an event +driven architecture. + +It means every meaningful action on mailboxes or messages triggers an event for any component to react to that event. + +`MailboxListener` allows executing actions upon mailbox events. They could be used for a wide variety of purposes, like +enriching mailbox managers features or enabling user notifications upon mailboxes operations performed by other devices +via other protocol sessions. + +Interactions happen via the managers (`RightManager`, `MailboxManager`, `MessageManager`, `MessageIdManager`) which emit an +event on the `EventBus`, which will ensure the relevant `MailboxListener`s will be executed at least once. + +`MailboxListener` can be registered in a work queue fashion on the `EventBus`. Each work queue corresponds to a given +MailboxListener class with the same configuration, identified by their group. Each event is executed at least once +within a James cluster, errors are retried with an exponential back-off delay. If the execution keeps failing, the event + is stored in `DeadLetter` for later reprocessing, triggered via WebAdmin. + +Guice products enable the registration of additional mailbox listeners. A user can furthermore define its own +mailboxListeners via the use of `extension-jars`. + +MailboxListener can also be registered to be executed only on events concerning a specific entity (eg. a mailbox). The +`registrationKey` is identifying entities concerned by the event. Upon event emission, the manager will indicate the +`registrationKey` this event should be sent to. A mailboxListener will thus only receive the event for the registration +key it is registered to, in an at least once fashion. + +## Consequences + +We need to provide an `In VM` implementation of the EventBus for single server deployments. + +We also need to provide [a distributed event bus implementation](0037-distributed-eventbus.md). + +## Current usages + +The following features are implemented as Group mailbox listeners: + + - Email indexing in Lucene or ElasticSearch + - Deletion of mailbox annotations + - Cassandra Message metadata cleanup upon deletion + - Quota updates + - Quota indexing + - Over Quota mailing + - SpamAssassin Spam/Ham reporting diff --git a/src/adr/0037-distributed-eventbus.md b/src/adr/0037-distributed-eventbus.md new file mode 100644 index 0000000..eea132e --- /dev/null +++ b/src/adr/0037-distributed-eventbus.md @@ -0,0 +1,46 @@ +# 37. Distributed Event bus + +Date: 2020-05-25 + +## Status + +Accepted (lazy consensus) + +## Context + +Read [Event Bus ADR](0036-eventbus.md) for context. + +Given several James servers, we need them to share a common EventBus. + +This: + - Ensures a better load balancing for `group mailbox listners`. + - Is required for correctness of notifications (like IMAP IDLE). + +## Decision + +Provide a distributed implementation of the EventBus leveraging RabbitMQ. + +Events are emitted to a single Exchange. + +Each group will have a corresponding queue, bound to the main exchange, with a default routing key. Each eventBus +will consume this queue and execute the relevant listener, ensuring at least once execution at the cluster level. + +Retries are managed via a dedicated exchange for each group: as we need to count retries, the message headers need to +be altered and we cannot rely on rabbitMQ build in retries. Each time the execution fails locally, a new event is emitted +via the dedicated exchange, and the original event is acknowledged. + +Each eventBus will have a dedicated exclusive queue, bound to the main exchange with the `registrationKeys` used by local +notification mailboxListeners (to only receive the corresponding subset of events). Errors are not retried for +notifications, failures are not persisted within `DeadLetter`, achieving at most once event delivery. + +## Related ADRs + +The implementation of the the distributed EventBus suffers from the following flows: + + - [Removing a configured additional MailboxListener](0026-removing-configured-additional-mailboxListeners.md) + - [Distributed Mailbox Listeners Configuration](0035-distributed-listeners-configuration.md) also covers more in details + topology changes and supersedes ADR 0026. + +The following enhancement have furthermore been contributed: + + - [EventBus error handling upon dispatch](0027-eventBus-error-handling-upon-dispatch.md) \ No newline at end of file --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
