chibenwa commented on a change in pull request #833: URL: https://github.com/apache/james-project/pull/833#discussion_r785721283
########## File path: src/adr/0053-email-rate-limiting.md ########## @@ -0,0 +1,70 @@ +# 53. Email rate limiting + +Date: 2021-01-10 + +## Status + +Accepted (lazy consensus). + +Not yet implemented. + +## Context + +Rate limiting is one of the common features expected from an email system. Examples: SaaS is +one https://www.fastmail.help/hc/en-us/articles/1500000277382-Account-limits#sending/receiving +, https://support.google.com/mail/answer/22839 + +They limit how many emails users can send/receive from/to each email account over a given period of time. +We believe the rate-limiting will help James to have more benefits: + +- Control of the resources +- Easy to configure dynamically the user policy. +- Complements the storage quota +- Can be a security requirement for SaaS deployments. +- Minimise impacts of Open-relay types of compression. +- Limiting the amount of emails sent to third parties can also prevent them from considering you as an open relay and can + be beneficial as well. + +## Decision + +Set up a new maven project dedicated to rating limiting. This allows the rate limiting mailets to be embedded in a James +server as a soft dependency using the external-jar loading mechanism. Please note that this will take the form of an +extension jar, that could be dropped in one's James installation, and thus is optional, and not a runtime dependency. + +Rate limiting will be enabled per sender, per receiver and globally. For each we will provide options for email size and +email count. + +- This can be done via mailets: + - PerSenderRateLimit is per sender + - PerRecipientRateLimit is per recipient + - GlobalRateLimit is for everyone. Depending on the position in the pipeline this could allow rate limiting all emails in + transit, relayed emails or locally delivered emails. + The rate limit will be configured + in [mailetcontainer.xml](/server/apps/distributed-app/sample-configuration/mailetcontainer.xml). + +- Those mailets will be based on a generic RateLimiter interface. We will propose two implementations for it: + - In memory (guava based) suitable for single instance deployments + - [Redis](https://redis.io) based, suitable for distributed deployments. + +The implementation chosen will be configurable as part of mailet configuration. One would be able to configure the +implementation he wishes to use. + +- We will document such a setup, and provide a mailetcontainer sample file. + +## Consequences + +- When having a change in the rate limit configuration, we need to restart James. +- Only protocols allowing to submit emails make sense here so SMTP and JMAP. +- It is more than acceptable to lose all redis data, which is equivalent to resetting the rate limiting. + +## Alternatives + +Alternatives implementation of the rate limiter can be proposed, and used within the aforementionned mailet. + +For instance one could rely on Cassandra counters and Cassandra time series (thus not needing additional dependencies) however we fear the potential performance impact doing so. Streaming based options, that aggregate in memory counters, might be a viable option too. + +## References + +- [JIRA](https://issues.apache.org/jira/browse/JAMES-3693) +- [Redis driver](https://github.com/lettuce-io/lettuce-core#reactive-api) +- [Guava rate limiter](https://guava.dev/releases/19.0/api/docs/index.html?com/google/common/util/concurrent/RateLimiter.html) Review comment: Hi Quan! This is a very good comment! > I read the Guava Rate limiter docs and I do not think its API support rate-limiting by other time units than Second directly. Yes this could be problematic as saying "I allow you 100 mails per hour" is not the same as "I allow you one mail every 36 seconds". Also the Guava rate limiter is blocking which is not really fancy. > Should we consider other libraries? We should ALWAYS look for alternatives! > It provides 2 implementations: Inmemory and Redis. The interesting thing is its Redis implementation uses lettuce-io reactive Redis client. So IMO we could benefit it reducing some Redis client code and counters implementation also. This is of course very interesting! Yet there is one key point to me: We need a James interface in plain Java (or Scala) fully abstracting away the RateLimitJ that would then just be an implementation detail (for memory AND redis implementations). Also the concept of "batching rules together" is interesting. We should definitly add this concept to our rate limiter API as well ( ``` RedisRateLimiterFactory factory = new RedisRateLimiterFactory(RedisClient.create("redis://localhost")); Set<RequestLimitRule> rules = new HashSet<>(); rules.add(RequestLimitRule.of(Duration.ofSeconds(1), 10)); rules.add(RequestLimitRule.of(Duration.ofSeconds(3600), 240).withPrecision(60)); ReactiveRequestRateLimiter requestRateLimiter = factory.getInstanceReactive(rules); Mono<Boolean> overLimit = requestRateLimiter.overLimitWhenIncrementedReactive("ip:127.0.1.6"); ``` ) @quantranhong1999 can you propose changes to this ADR following your remarks? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
