Re: Questions about Guice design
> A hexagon defines some ports (the component it needs to work), some adapters > (the implementations it provides to other hexagons) and a core domain: the > logic of this hexagon. Thank you for elaborating your thoughts. I see much better now what you are trying to do. I like many of the ideas of the hexagonal architecture that you are trying to achieve. In that case, let’s just focus on the core. The rest is really not that important at this time if the immediate goal is to document the project. I am a long-time enthusiast of DDD. If the “core” of the hexagonal architecture matches the “core domain” in the DDD sense, it would make the system very easy to understand. The “core” domain ought to be very explicit and should exactly match the domain concepts. When reading the code, it ought to be really obvious what is happening. My impression is that we agree on pretty much everything except maybe the details about how the core is developed. I don’t think it’s important at this time to go into that, though. From where I am coming from, in general the core domain should be able to pop out to somebody like me with very little effort. The ports and adapters merely provide access to the core (or support for the core, depending on your perspective), so if the core is clear and well understood, then the rest ought to be quite secondary. When using the system, the core does not change, only the “outside” components that interact with the core change. For this reason, the wiring that happens (for example with Guice Modules) is basically the connection between the outside components to the core via the ports/adapters, but the ports/adapters ought to be obvious if the core communicates its intentions well. If that describes James, then the path forward is now clear. > `Core domains` are a bit hidden in the current code With your help, I will try to figure out what the core is, and the rest should follow quite nicely. I will start a separate thread relating to the “core”, because this topic has drifted from the original discussion about Guice. Unless you think I am way off track with your thinking, I would suggest that you not reply to this email, but wait for me to start the new thread. :-) I think it’s good that we got here. Thanks for your patience. Cheers, =David - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: Questions about Guice design
I realize while looking for resources about hexagonal architecture that I probably used port/adaptor words differently from what they are supposed to mean. I'll ask some friends to correct me on that matter. On Tue, 2020-05-19 at 20:41 +0900, David Leangen wrote: > Hi Matthieu, > > [...] > > > * `ports` that define some APIs required for `domain cores` > > (mailbox- > > api is a port) > > Ok, this is interesting because it is exactly the opposite approach > from what I am used to. :-) > > I am used to designing the API first. Since the API is exposed to the > outside, it usually requires a lot of thought. Then an implementation > gets built that satisfies the API. Since the implementation is > “private”, it can be changed or updated whenever necessary (or there > can even be multiple implementations at once), so it doesn’t require > quite as much thought as the API. Definitely the API should remain as > stable as possible, as changing it affects all of its users. > > It sounds from your description that the implementation (core) is > done first, then the API is more of an afterthought. > > Or did I misunderstand? We have the same definition of what an API and an implementation is. It's the reason why APIs have contract testsuites: they are important and require stability. What you miss in the picture, and I guess you should read more about hexagonal architecture, is that a core domain is a component that is written in pure Java (no technical dependency) that encapsulate the logic of a given domain. The `port` concept is here to say "I will need to store that email somewhere, so I define an API because I don't care how it works as long as it works with plain Java objects. Please write an implementation of that API and give it to me". > > * `adapters` that provides the implementation of a `port` (mailbox- > > cassandra) > > But if mailbox is a “core” component implementation, that has an > adapter, and also has multiple other implementations… Sorry, I’m a > bit confused. Maybe an image would help? An hexagon defines some ports (the component it needs to work), some adapters (the implementations it provides to other hexagons) and a core domain: the logic of this hexagon. Note that an hexagonal `core domain` can very well be just `infrastructure` (adaptor) for another hexagon. It's not layers but a recursive architecture. And sorry, I can't find illustration about that. > > > When we define a `port` API we provide a contract testsuite to > > check > > that the implementations will work well plugged-in. > > Yeah… like I said, this is backwards from what I am used to. :-) > > My fear is: if the API is just an afterthought that is placed over > the implementation, how do you know if it was well designed? Because it fits the `core domain`. The `core domains` are what matters in a software, everything else is just technical details. > > > So to come back to the analysis: the right level of abstraction for > > James architecture is these three entities. > > Those sounds like very abstract architectural entities to me, > something like a “database table” or a “queue” or some other general > construct. Or am I misunderstanding? > > In my mind, if you were to say that they are just “categories” of > actual James entities, perhaps that would make more sense to me. For > example: “A Mailbox is a core entity in the James domain, and happens > to be one of the core entities” makes sense to me, but it is the > Mailbox that is the entity, not “a core” that is an entity. Does that > make sense? Or again, am I completely misunderstanding what you are > trying to say? > > > > Unfortunately, there's not way to visualize the hexagons from the > > code > > right now. > > :-) > > Ok, that should be our objective I guess, then. The code should match > the domain model, so it should be easy to find our way around in the > code. `Core domains` are a bit hidden in the current code (it can be an abstract class for example) so the easiest way to draw our hexagons is to review *-api projects: they describe the boundaries between hexagons. It could be fun to draw the hexagons for various profiles actually. > > > Finally, I see Guice module documentation as a problem: we usually > > document things that are supposed to last. Documentation reduces > > the > > ability to refactor things. > > Yes, that makes sense. Really it should be the API that is > documented, as normally (again from my experience) the API is the > representation of the domain model in code. APIs are just boundaries. That means it can be the boundary for the user facing features but also the boundary between your imap protocol and your mailbox `core domain`. > > > As Guice modules are technical classes, I would not like to be > > refrained from refactorings because of this documentation. > > Completely agree!! > > > > It's why I think we have to find a way to document `ports` and > > `adapters` because they are definitely more
Re: Questions about Guice design
Hi Matthieu, >> I would wager that in most situations, this is the level of abstraction that >> is most useful for a developer or system operator (and definitely for an >> application assembler). The only time a package or class level is necessary >> is when actually making changes to the code. > Let me disagree. Yes, please. :-) > Guice is a technology we use but it is not mandatory, we could change > this part of James tomorrow if we want, it's an implementation concern. I completely agree with this statement. Just to show to what extent I agree, allow me to even add to your statement. It could be Spring, OSGi, or whatever. It just happens to be Guice. Guice isn’t even the implementation, it is just the framework code that wraps the implementation, so I would argue that “Guice” is even lower-level and less important than what you are stating. I have no argument at all with this idea. The only point of Guice, as far as I can tell, is to inject implementations into APIs. If it were in OSGi, it would be the service object. I am sure that the principle is much the same in most frameworks. The reason I was dwelling on the Guice Components is because I am unable to figure out what the API of the domain model is, and the API interfaces should have a very clear 1:1 correspondence with the James model. (Actually, I usually tend to think that the API **is** the model, expressed in code.) Since I am having a hard time with understanding the code base, I thought I could use the Guice Components as a proxy for the domain model. My thinking is that in general terms, given a domain concept, there will be one or more interfaces, and for each of these, there will be an implementation. For each of those, it will be wrapped as a Guice Module. So by understanding the Guice Modules, I should be able to work my way back to the domain model and eventually understand what is going on in the code. So I thought it was a safe assumption to give me at least a starting point as to where to eat this elephant. But reading below, I see that our ideas of what is API vs. implementation seem to be a little different... > Basically, you have: > > * `domain cores` (say managing emails and mailboxes, managing users) > that implement the important domain logic Ok, sounds good. If I could understand better what these “cores” are and where they are represented in the code, that would already give me a start. > * `ports` that define some APIs required for `domain cores` (mailbox- > api is a port) Ok, this is interesting because it is exactly the opposite approach from what I am used to. :-) I am used to designing the API first. Since the API is exposed to the outside, it usually requires a lot of thought. Then an implementation gets built that satisfies the API. Since the implementation is “private”, it can be changed or updated whenever necessary (or there can even be multiple implementations at once), so it doesn’t require quite as much thought as the API. Definitely the API should remain as stable as possible, as changing it affects all of its users. It sounds from your description that the implementation (core) is done first, then the API is more of an afterthought. Or did I misunderstand? > * `adapters` that provides the implementation of a `port` (mailbox- > cassandra) But if mailbox is a “core” component implementation, that has an adapter, and also has multiple other implementations… Sorry, I’m a bit confused. Maybe an image would help? > When we define a `port` API we provide a contract testsuite to check > that the implementations will work well plugged-in. Yeah… like I said, this is backwards from what I am used to. :-) My fear is: if the API is just an afterthought that is placed over the implementation, how do you know if it was well designed? > So to come back to the analysis: the right level of abstraction for > James architecture is these three entities. Those sounds like very abstract architectural entities to me, something like a “database table” or a “queue” or some other general construct. Or am I misunderstanding? In my mind, if you were to say that they are just “categories” of actual James entities, perhaps that would make more sense to me. For example: “A Mailbox is a core entity in the James domain, and happens to be one of the core entities” makes sense to me, but it is the Mailbox that is the entity, not “a core” that is an entity. Does that make sense? Or again, am I completely misunderstanding what you are trying to say? > Unfortunately, there's not way to visualize the hexagons from the code > right now. :-) Ok, that should be our objective I guess, then. The code should match the domain model, so it should be easy to find our way around in the code. > Finally, I see Guice module documentation as a problem: we usually > document things that are supposed to last. Documentation reduces the > ability to refactor things. Yes, that makes
Re: Questions about Guice design
Hi David, On Sun, 2020-05-17 at 15:01 +0900, David Leangen wrote: > Hi! > > [...] > Organizing into Guice Modules is very nice. One of the important > benefits is that it helps a lot to break the system down into > reasonable chunks that are more understandable. It is simply > impossible for a normal human to understand the system all at once. > It is necessary to have several levels of abstraction. The use of > Guice Modules to provide a “just right" level of abstraction (not too > high so as to be useless when trying to run it, but not so low as to > be much too detailed) is essential. I would wager that in most > situations, this is the level of abstraction that is most useful for > a developer or system operator (and definitely for an application > assembler). The only time a package or class level is necessary is > when actually making changes to the code. Let me disagree. Guice is a technology we use but it is not mandatory, we could change this part of James tomorrow if we want, it's an implementation concern. What makes James special is the hexagonal(-ish) architecture it uses. I won't go into details here about why it's not exactly how it should be but to understand James you have to understand this design. Basically, you have: * `domain cores` (say managing emails and mailboxes, managing users) that implement the important domain logic * `ports` that define some APIs required for `domain cores` (mailbox- api is a port) * `adapters` that provides the implementation of a `port` (mailbox- cassandra) When we define a `port` API we provide a contract testsuite to check that the implementations will work well plugged-in. So to come back to the analysis: the right level of abstraction for James architecture is these three entities. Unfortunately, there's not way to visualize the hexagons from the code right now. Finally, I see Guice module documentation as a problem: we usually document things that are supposed to last. Documentation reduces the ability to refactor things. As Guice modules are technical classes, I would not like to be refrained from refactorings because of this documentation. It's why I think we have to find a way to document `ports` and `adapters` because they are definitely more stable. Cheers, -- Matthieu Baechler - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: Questions about Guice design
On 18/05/2020 16:16, David Leangen wrote: > > Thank you for the explanations. I now understand much better the migration > from Spring to Guice, and the intent behind Guice. > > Instead of directly answering the questions you posed, I will make a proposal > that I think goes in the same direction as your comments. > >> This documentation effort would be a great move toward James as a >> toolkit to write your own email server. We of course had it in mind >> during Guice adoption, but not yet had feedback on the topic. > > Although quite tedious, I propose as a first step that I simply ask questions > about each of the 144 Guice Modules in the system. I will take notes in the > form of Javadoc in order to keep the information together with the code. Just > knowing what they are intended for will be a good start. Once each of them is > documented, I think it should be easier to analyze what to do with it. > > I am sure that in some cases, maybe even many, I should be able to guess > about the purpose of the Module. So to save time, I will try, as much as > possible, to propose an explanation and ask for feedback. > > When I don’t understand, I’ll have to ask for help. > > In any case, this tedious project should be a big step forward, but it > depends on whether or not the developers here are willing / able to help. :-) > > > Please let me know if I should or should not continue in this direction. Please continue. That is an awesome proposition! I am available to answer Guice related question and promote James usage as a toolkit. Cheers, Benoit > > > Cheers, > =David > > > > - > To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org > For additional commands, e-mail: server-dev-h...@james.apache.org > - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: Questions about Guice design
Thank you for the explanations. I now understand much better the migration from Spring to Guice, and the intent behind Guice. Instead of directly answering the questions you posed, I will make a proposal that I think goes in the same direction as your comments. > This documentation effort would be a great move toward James as a > toolkit to write your own email server. We of course had it in mind > during Guice adoption, but not yet had feedback on the topic. Although quite tedious, I propose as a first step that I simply ask questions about each of the 144 Guice Modules in the system. I will take notes in the form of Javadoc in order to keep the information together with the code. Just knowing what they are intended for will be a good start. Once each of them is documented, I think it should be easier to analyze what to do with it. I am sure that in some cases, maybe even many, I should be able to guess about the purpose of the Module. So to save time, I will try, as much as possible, to propose an explanation and ask for feedback. When I don’t understand, I’ll have to ask for help. In any case, this tedious project should be a big step forward, but it depends on whether or not the developers here are willing / able to help. :-) Please let me know if I should or should not continue in this direction. Cheers, =David - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: Questions about Guice design
Hi David! A pretty long and interesting email! On 17/05/2020 13:01, David Leangen wrote: > > Hi! > > As I just mentioned in a different email: > >> As part of my documentation project, I am starting to turn my attention >> towards the code itself. I am trying to understand how the system works. […] >> To understand the system better, I need to take a step back and understand >> Guice. I am learning about Guice now. So far, guice looks quite easy to >> understand and use (at least for me because as a long-time OSGi user I >> understand very well the concepts of DI, api/impl separation, services, >> etc.). I can understand very well the motivation for using a framework like >> Guice, and I think (hope!) it should help me to understand what is going on >> in the system. > > You’ll have to excuse my inexperience with Guice. However, to understand > James, it is imperative (ok, actually “declarative” hahahaha I’m so funny) to > understand its organization into Guice Modules. I have not read through all > of the Guice documentation, but I think I get the idea. > > In this context, I have started looking at the James code base. There are > many very nice things about it. Despite its size and complexity, I was able > to understand quite a lot thanks to the good organization and naming > conventions. That said, I think we could still do a lot better. > > Organizing into Guice Modules is very nice. One of the important benefits is > that it helps a lot to break the system down into reasonable chunks that are > more understandable. It is simply impossible for a normal human to understand > the system all at once. It is necessary to have several levels of > abstraction. The use of Guice Modules to provide a “just right" level of > abstraction (not too high so as to be useless when trying to run it, but not > so low as to be much too detailed) is essential. I would wager that in most > situations, this is the level of abstraction that is most useful for a > developer or system operator (and definitely for an application assembler). > The only time a package or class level is necessary is when actually making > changes to the code. > > I would even double down on my statement and say that the organization of the > Guice Modules is perhaps THE most important abstraction available to allow > people to understand the system. I had a write at "why we chose Guice" and your statements perfectly complete it, I will add it in https://github.com/apache/james-project/blob/master/src/adr/0036-against-use-of-conditional-statements-in-guice-modules.md > > Ideally, to help provide a better understanding of the system and its > compile-time (and even to some extent its runtime) organization, I think it > is important to: > > * Match the Guice Modules with the Maven modules, matching them exactly if > possible > —> At first glance, this seems pretty good That is what we tried to do, however we also wanted to limit the cardinality of maven modules, and split guice-maven-modules when it was used by different profiles. > > * Ensure that each Module is well-contained (i.e. no “leaks” or coupling to > other implementations) > —> I found this part to be quite problematic (topic for a different > email) > > * Understand the API surface area of each Module > —> I am having a lot of trouble with this (again, a topic for a > different email) > > > Below is a list of all the Guice Modules I was able to find in the system. I > simply did a search and manually extracted these. There are 144!! > > Although I could guess a little bit as to what they do, I was otherwise > unable to understand many things: > > * What is the purpose? I don't understand this question. If you take into account the cardinality (profile x components) and the fact that profiles don't enable the same administration features (that thus needs to be split), a high cardinality is in my opinion understandable. > * How does it relate to other Modules? > * Which Modules are implementing the same API, so are “swappable”? > * What is the hierarchy? (Assuming that there are Modules of Modules)? Modules can "install" other modules, we use that to better split the logic, for instance for the JMAP protocol. > > > It is a great start! However, to be able to put more order into all of this, > these are my initial thoughts: > > 1. Document each module individually (even if only a line or two of text) > 2. Understand if there is any hierarchy (i.e. Modules of Modules) > —> Refactor / Rename the Modules to make the hierarchy immediately clear > 3. Validate that each Guice Module is perfectly aligned with its Maven module > 4. With this clearer picture, evaluate whether or not the current project > structure is still appropriate > 5. Update the documentation of each Maven module to show all of its Guice > Modules This documentation effort would be a great move toward James as a toolkit to write
