On 22/05/2020 18:06, David Leangen wrote:
> Thanks Benoit, very helpful! It does lead to more questions, though. :-)
The contrary would have surprised me ;-)
I have been yet again quitte verbose once again, don't hesitate to raise
a warning if the signal/noise ratio gets to low...
Cheers,
>
>> - **mailet** are separated in an effort to serve as a specification for...
>> - **mailbox** as presented below is a storage service for the server….
>> - **protocols** is an effort to provide mail protocol utility for...
>> I hope I provided some context about the project structure…
>> I would call the core component the **mailet container**...
>
>
> Yes, indeed! Let me try to rephrase this in my own wording to test if I
> understood. I’ll use the Servlet spec as an analogy, because what you write
> about Mailet and protocols triggered that comparison in my mind.
>
> If we use the analogy of the HTTP<—>Servlets, then in this case there would
> be a similar relationship between for example IMAP<—>Mailet.
>
> A gross simplification of the essence of a generic Servlet is:
>
> ```
> public interface Servlet {
> void service(Request req, Response rsp) throws ServletException;
> }
> ```
>
> More specific to the HTTP protocol is:
>
> ```
> public class HttpServlet {
> public void get(HttpRequest req, HttpResponse rsp) throws ServletException;
> public void put(HttpRequest req, HttpResponse rsp) throws ServletException;
> public void post(HttpRequest req, HttpResponse rsp) throws ServletException;
> public void delete(HttpRequest req, HttpResponse rsp) throws
> ServletException;
> public void etc(HttpRequest req, HttpResponse rsp) throws ServletException;
> }
> ```
>
>
> So for a Mailet, the idea would be:
>
> ```
> public interface Mailet {
> void service(Request req, Response rsp) throws MailetException;
> }
> ```
>
> And for the IMAP protocol:
>
> ```
> public class ImapMailet {
> public void doSomethingImapSpecific(IMAPRequest req, IMAPResponse rsp)
> throws MailetException;
> }
> ```
Mailet is intended for a specific goal: Mail processing.
Regarding IMAP, the components processing IMAP commands are ImapProcessor.
>
> However, the actual generic signature is:
>
> ```
> void service(Mail mail) throws MessagingException;
> ```
>
> (By the way, does this mean that the Mail is mutable and that the service
> method creates a side-effect by modifying the Mail object??)
Yes.
(To be honest, in the comparison you do a servlet is doing a side effect
to send a response).
That 'mutation' is due to the lack of efficient immutable data-structure
to represent a mime object (long term goal of the mime4j library).
>
> Is the analogy something along the right lines? So for example: "Tomcat is to
> the Servlet spec as James is to the Mailet spec"??
>
>
> Even if the analogy makes sense, the one big difference is that Servlets are
> stateless, but emails are not, so as you pointed out it needs the concept of
> Mailboxes for storage.
There is a lot to say about this sentence.
Most (all?) mailets in James are stateless.
The mailbox is needed even with an immutable processing. As it
represents the mails that a given user received, so that he can read
them when he wishes.
The "storing an email in a mailbox" is a business rule, not a technical
limitation.
Note that mailbox is not a hard dependency for the mailet-container. It
is defined via a plugin (a LocalDelivery mailet) and you can very well
introduce a profile running completly without it. We did it in a SMTP
only James profile tested here [1] (not exposed to end user, but rather
intendeed to demonstrate James as a toolkit to build email server)
[1]
https://github.com/apache/james-project/blob/master/mpt/impl/smtp/cassandra-rabbitmq-object-storage/src/test/java/org/apache/james/mpt/smtp/CassandraRabbitMQAwsS3SmtpTestRuleFactory.java
>
>
>>> * Mime4j
>>> * jSieve
>>> * jSFP
>>> * jDKIM
>>
>> I would call them "external librairies".
>
> To me, according to the vocabulary that I am used to (meaning that I am not
> claiming to be “right”, just that this is my experience), a "library" is
> something that gets embedded into a "component". Both are just ways of
> packaging up code, but the library is both “hidden” and "disposable" (meaning
> that it is just part of the implementation and is not seem from the outside,
> nor cared about), while the "component" is something that is actually seen
> from the outside and worth understanding in order to understand the entire
> system.
>
> If users or maybe even developers of James don't have to care about Mime4j
> and it can be hidden away, then I would agree that it is a library.
So far I would agree with that statement.
> It would just be part of some other components, and would only come into
> scope when developing that component. So for this documentation project, I
> probably wouldn't even have to mention it at all.
>
> Or in other words, only one of the following is correct:
>
> * These concepts are understood internally by James core, but the
> implementation code is provided by these libraries and does not need to be
> exposed to the outside
>
> OR
>
> * These concepts are not understood by the James core, so both the concepts
> (API) and the implementation are provided by these external projects
Sorry if I did not expressed myself correctly
By "library" I mean that these projects are used outside their James
context.
They offer a common service that java projects working with emails might
be interested in.
See for instance this reaction:
http://mail-archives.apache.org/mod_mbox/james-mime4j-dev/202005.mbox/%3C16B1D214-B4E6-4230-96AC-CEDCD2D0D934%40whiuk.com%3E
This might be seen also as an effort to "advertise" the James ecosystem.
>
>
>>> Administration
>>> ------------------------------------
>>> The port is the “Admin API”. There are 3 adapters: JMX, REST, and CLI.
>>
>> A user could be loading additional webadmin routes (~plugin)
>
> Sorry, I don’t understand this statement…
Sorry.
As a user running James, I can write my own custom WebAdmin API routes
and load them into a running James server.
This will result in these routes being expose within that James server.
So "webadmin routes" is a plugin, a "port" a developer can use to modify
James behaviour (somehow just like mailets)
>
>
>
>>> Storage API
>>> ------------------------------------
>>> This is actually a general term for 3 APIs:
>>>
>>> * Mailbox API
>>> * Search API
>>
>> Which is called "search API" is in my opinion a sub-part of the
>> mailbox-api.
>
> Ok. As for the rest of of the details you provided about the mailbox-api,
> I’ll get back to that later.
>
>
>>
>>> * User API
>>
>> * data-api allow persistence of server data like users (above
>> mentionned User API), domains, recipient rewriting rules. There might be
>> some work to better name this component, and refine its boundaries.
>>
>> There is I think much more going on at the storage level.
>
> But you already wrote above that it is Mailbox that describes storage in the
> context of a Mailet-based system like James.
No.
mailbox-api is loosely related to mailet and mail processing.
Storing a "message" in a mailbox is a potential result of processing a mail.
Again the entities needs to be more clearly defined.
- A mail is a MIME message in transit (with a sender and recipients)
- A message belongs to a user, who can read it, retrieve it, delete it,
mark it as read, etc.
A mailbox stores message, not mails.
>
> If something is being stored, then it must be because the concept is known to
> the core part of the system.
Yes.
The LocalDelivery mailet is acting as an adapter between the
mailet-container component (where emails flows) and the mailbox component.
A nice diagram might be http://james.apache.org/images/james-imap-server.png
> Or are you saying that the core offers a generic “data storage” service to
> outside components that want to use it? So it is some generic storage service
> like labels in Kubernetes that is used however external libraries want to use
> it?
>
>
>> * mailQueue is clearly a central component is James (as in any SMTP server)
>
> But I have understood from all of the documentation and our emails so far
> that "SMTP is just a protocol". It sounds like there is more to it than just
> a protocol, then. There are expectations about behaviour and the services
> offered by the system.
SMTP defines the fact that emails gets asynchronously processed. If not,
you have the "local" mail transfer protcol (LMTP)
>
>
>> * blob-store is a component for storing potentially large binary data.
>> Other core-components implementations can rely on it to store
>> (potentially large) binary data
>
> How is this related to Mailbox? Does Mailbox use the blob store (so Mailbox
> is like the Aggregate Root if this were a DDD concept)? Or does Mailbox get
> bypassed, so that actually storage is more like Mailbox+blobstore?
A mailbox implementation *might* use the blobStore component.
The cassandra mailbox does this, not the JPA one.
(Blob Store is not a top level component but rather a utility other
components can use)
>
>
>> * mail-repository allows storage of email with their processing context
>
> So you mean it adds extended functionality to a Mailbox? I’m not entirely
> sure I understand this statement.
The entity is not the same.
Mailbox stores messages belonging to a user, who can read it, retrieve
it, delete it, mark it as read, etc.
MailRespository stored mails (with their envelope) that we don't want to
flow anymore. It could be used upon error, spam, etc... An administrator
can reprocess them anytime he wishes.
>
>
>> * sieve script storage (nowadays part of data-api)
>
> Right, but I already understood that sieve is “external”, so now I am
> confused.
JSieve is a library helping with parsing Sieve scripts.
It has some external usages (Zimbra), and lives so far as a standalong
library.
Now in James we have the following features related to Sieve:
- As a user I want to store sieve scripts on the server
- As a user I want my Sieve scripts to be executed upon mail reception
So we need some extra components in James in addition to the Jsieve
library to make this work:
- The webAdmin routes to manage Sieve scripts
- The Sieve management mailet (ManageSieveMailet)
- The managesieve protocol (experimental)
- Sieve script storage.
These services goes way further than just the Sieve script parsing
offered by JSieve.
Clearer?
>
>
>
>> Task manager
>> ---------------------
>> Allows to control and schedule long running tasks run by other components.
>> We should, in my opinion, allow additional task registration via plugins.
>
> So there is a core concept of a “task”? Is this a cross-cutting concern that
> is known by all components of the system? So for example the Mailbox admin
> knows that “Move Mailbox” is a task, and therefore it can be scheduled?
>
> Can you please explain more about what a “task” is in James, and how it is
> used?
Same than blobStore. It is more of a utility that other components might
rely on.
To give you some example:
- Re-processing some emails stored in a mail repository is a task.
- Removing some mails in the mail queue is a task.
- Migrating data between two Cassandra tables is a task.
All these core-components have similar needs (cancellation, monitoring
of long running jobs) that the taskManager helps to fullfill.
>
>
> Cheers
As a conclusion, we might rely benefit from documenting the core
entities core components acts on, I think it could clearly ease your
understanding.
Also we should focus on core components first, utility components
(blobStore, taskManager) are likely less useful for the big picture
understanding. Sorry for the digression.
> =David
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
> For additional commands, e-mail: server-dev-h...@james.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
