Hello Jean
On 01/03/2024 14:01, Jean Helou wrote:
Hello Benoit
This is very clear thanks. I was not implying you should use pulsar :) I
use a sass deployment and I am aware it is quite complex to operate:)
I might well end up deciding that a pulsar+redis makes more sense than
pulsar only when I start trying to deploy the MDA side of James:)
I like that flexibility!
Combining different technologies for queuing and pubsub likely make
sense, but maybe we would need to have more "generic" interfaces under
the hood to make those combinations easier to achieve.
If you are interested we could plan a "pair programming" session on the
topic...
- Currently the James domain logic would imply one topic per mailbox
(more or less) and we would too quickly IMO end up with an unconfortable
number of topics (million+). A refactoring would be needed.
Hmm this is interesting. I will have to review if it would be problematic
for a pulsar implem do you have a good entry point (class name(s)) I should
look at to investigate ?
Well the problem is more linked to the use of the eventbus than the
event bus itself.
https://github.com/apache/james-project/blob/master/mailbox/api/src/main/java/org/apache/james/mailbox/events/MailboxIdRegistrationKey.java
https://github.com/apache/james-project/blob/d3ec938e0aab741b79c9199e6118ee5464ead646/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreMessageManager.java#L316
Thanks again for the explanation
Cheers
Jean
Note that all of that makes it a Linagora topic, at least now. If
relevant, we would happily contribute it back ;-)
Is this more clear?
Best regards,
Benoit TELLIER
On 01/03/2024 13:17, Jean Helou wrote:
Hello 👋
I'm not (yet) familiar with the event bus :)
Out of curiosity could you describe what made you chose redis and what
other systems you considered ?
What characteristics you thought would match well with the problem.
I'm interested in part because there is a Jira to implement the event bus
over pulsar so I'd like to take the opportunity to better understand the
design ideas behind the event bus as that would help with the pulsar
implementation.
Jean
Le mer. 28 févr. 2024 à 10:09, Quan tran hong <
quan.tranhong1...@gmail.com>
a écrit :
Hi folks,
Following the context of the Jira ticket JAMES-3996 POC: Move RabbitMQ
Event bus user notifications to Redis
<
https://issues.apache.org/jira/projects/JAMES/issues/JAMES-3996?filter=allopenissues
(TLDR:
we observed some annoying issues with RabbitMQ in a deployment and we
think
it could be better to move at least the user notifications part of Event
Bus to Redis), I did a Proof Of Concept about that in the PR JAMES 3996
Redis event bus POC <https://github.com/apache/james-project/pull/2028
.
The POC is considered done to me, and I want to share the POC result to
James devs:
- It is *possible *(the POC worked!) to replace the Event bus user
notifications (key registration) using Redis Pub/Sub. And likely the
whole
EventBus (the group registration part as well) can rely on Redis (I did
not
do that part).
- I did several performance tests for the POC, and the results were
*good*.
Regarding the metrics of Event Bus user notifications, Redis seems to
outperform and show more stability in response time than RabbitMQ. (For
more details, I shared on the PR)
Despite the POC has been worked and shown some prospects, I think we
should
monitor it more carefully before applying it to James. Therefore, we
(Linagora) would adopt the Redis Event bus user notifications first in
our
TMail project (based on James). We would keep an eye on its stability
before contributing the fine-tuned work toward James.
By the way, it seems someone from the community is interested in
building
the whole EventBus using Redis cf
https://issues.apache.org/jira/browse/JAMES-3956. This POC could be a
start
for that Redis EventBus.
What do you think about using Redis for EventBus? Would that sound
interesting to you?
Regards,
Quan
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
